Reading a file into a vector all at once

Question

This is a good idea to read in std::vector , or there are some here:

 using namespace std; ifstream file("blah.dat", ios::binary); vector<T> v(N); file.read(static_cast<char*>(v.data()), N * sizeof(T)); 

Does the vector standard allow this to populate the vector? For simplicity, suppose T is the usual old data type.

Answer 1

There is no undefined behavior if T is trivially copied, which, of course, is POD. vector<T>::data guaranteed to return a pointer to a continuous array of vector<T>::size T s, and the object representation of a trivially copied type T guaranteed as a continuous sequence of sizeof(T) bytes (possibly including internal upholstery).

If the bytes that you store in this space are not allowed T represent objects, you can get undefined behavior when accessing them. Exactly what byte sequences represent the actual representation of the object T is a slightly gray area; at least you should be able to portablely write the base bytes of an object of a trivially copied type to a file and successfully read them back into the base bytes of an object of the same type.

For paranoia, I would probably put:

 static_assert(std::is_trivially_copyable<T>(), "NO NO NO - T MUST BE TRIVIALLY COPYABLE!"); 

before file.read for future verification.

Answer 2

It looks like you benefited from a memory mapped file. Boost provides two implementations, so you do not need to directly contact mmap() .

Using boost.iostreams :

 #include <boost/iostreams/device/mapped_file.hpp> boost::iostreams::mapped_file_source file("blah.dat"); std::size_t size = file.size() / sizeof(T); const T * ptr = reinterpret_cast<const T*>(file.data()); for (std::size_t i=0; i<size; ++i) std::cout << ptr[i] << std::endl; 

You can also use boost.interprocess , but this requires more code for almost the same functionality.

The main advantage is that you do not allocate any memory to access the file, it will load on demand when accessing the data. The data itself will live in cached / buffered pages, so it does not take up any memory (it is discarded if the system needs memory for something else.)

Answer 3

You should not do this, in fact what you should do (starting from the source C ++) is

 std::ifstream file("foo.txt"); file >> std::noskipws; // use this line so that whitespace won't be skipped std::vector<char> buffer(std::istream_iterator<char>(file), std::istream_iterator<char>()); 

The reason you shouldn't do it your own way is because it makes no sense for the object to live in a file (at least in C ++). Files contain only characters, which are then formatted using operator>> to create objects. The standard allows the compiler to do some really strange things for the object (especially when RTTI is turned on), which makes it useless to "save" the object in a file. You are much better off creating your own serialization format.