I am trying to run for-loop in R (2.15.0) using large files / matrices (64-bit Windows Vista, 4G RAM). If I do one iteration of this loop, it works fine. But if I started a cycle on several files (and would like to make hundreds of them), I ran out of memory. I tried to delete the files at the end of the loop, but it does not return memory in Windows (as I see in the Windows task manager). It seems that R is trying to get more and more memory from my OS, instead of using the internal memory after deleting the files. Is there a workaround for this? If you need more information about a research question, I will be happy to share the rest to find the right solution.
Thanks! Cheers, Robert
> library(VariantAnnotation) > fi<-list.files("E:/1000genomes/chr22",full.names=T) > for(i in 1:length(fi)) { + input=paste("smaller.00", i, ".gz", sep = "") + output=paste("geno.", i, ".RData", sep = "") + vcf = readVcf(input, "hg19") + genotypes=geno(vcf)$GT[,] + save(genotypes, file=output) + gc() + } Error: scanVcf: Realloc could not re-allocate memory (873600000 bytes) path: E:\1000genomes\chr22\smaller.002.gz In addition: Warning messages: 1: In doTryCatch(return(expr), name, parentenv, handler) : Reached total allocation of 3963Mb: see help(memory.size) 2: In doTryCatch(return(expr), name, parentenv, handler) : Reached total allocation of 3963Mb: see help(memory.size) > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 4543758 242.7 12363911 660.4 18010556 961.9 Vcells 19536404 149.1 61090604 466.1 119317584 910.4
and if I delete the contents at the end of my script:
+ save(genotypes, file=output) + rm(vcf) + rm(genotypes) + rm(input) + rm(output) + rm(getal) + rm(i) + } Error: scanVcf: Calloc could not allocate memory (18 of 1 bytes) path: E:\1000genomes\chr22\smaller.001.gz In addition: Warning message: In doTryCatch(return(expr), name, parentenv, handler) : Reached total allocation of 3963Mb: see help(memory.size) > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 2355472 125.8 10798339 576.7 16872405 901.1 Vcells 1992717 15.3 62280756 475.2 105556441 805.4
And I found on the Internet that it works from the command line, so I put the script in the file "runthis.R" in the R-root directory and ran: Rscript.exe runthis.R --no -save --no-restore It executed one additional file, and then reported the same error.