Check out the bigmemory package, as well as related packages such as bigtabulate , bigalgebra , biganalytics and others. There's also ff , although I don't find it user friendly, like a bigmemory . It is reported that the bigmemory package was partially motivated by the difficulty of using ff . I like this because there are very few changes for my code to access the bigmatrix object: it can be manipulated in much the same way as the standard matrix, so my code is very reusable.
HDF5 is also supported there via NetCDF4, in packages such as RNetCDF and ncdf . This is a popular multi-platform multilingual method for efficiently storing and accessing large datasets.
If you want to use the basic functions of memory mapping, look at the mmap package.
Iterator
source share