R-based memory fire event

Question

I was wondering what a good way to trigger a memory-based event in R. Let's say I am combining a bunch of files into one master file, but the size of the entire main file may be too large to hold in memory. When I come to the limit of memory, I would like to save the current current file and free memory.

master <- NULL
partnum <- 1
threshold  <- 0.8
filelist <- list.files(mypath)

for (filename in filelist) 
{
   filedata <- read.csv(filename)
   if (is.null(master)) master <- filedata
   else master <- rbind(master,filedata)
   rm(filedata)

   # test for memory usage here
   # if (usedMemory > availableMemory * threshold)
   # then do the following else go to top of loop

   save(master,file=paste(mypath,partnum,"rData",sep="."))
   master <- NULL
   partnum <- partnum + 1
}

, , . , . , script 10 , , 8 . , , 5 , , 4 .

> x <- 1:10^9
> memory.size()
[1] 3832.26
> memory.limit()
[1] 16381
> gc()
            used   (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells    164953    8.9     350000   18.7    350000   18.7
Vcells 500150216 3815.9  669246830 5106.0 550150069 4197.4

10 , 2

Answer 1

, memory.size. , - :

# Are we using more than 1 GB?
if (memory.size() > 1000) {
  # Force a garbage collect and check again
  gc()
  if (memory.size() > 1000) {
    # free up memory...
  }
}

memory.size , , ( ).