How to directly read a huge chunk of memory in std :: vector?

Question

I have a huge x continuous array that I fread from a file.

How to dump this piece in std::vector<> ? In other words, I prefer the result to be in std::vector<> rather than in an array, but I want the resulting C ++ code to be as efficient as this simple C version, which throws a piece directly into an array.

From the search around, I think I may have to use a new placement in one form or another, but I'm not sure about the sequence of calls and problems with ownership. Also, do I need to worry about alignment problems?

I am testing with T = unsigned , but I expect a reasonable solution to work for any POD structure.

 using T = unsigned; FILE* fp = fopen( outfile.c_str(), "r" ); T* x = new T[big_n]; fread( x, sizeof(T), big_n, fp ); // how do I get x into std::vector<T> v // without calling a gazillion push_backs() or copies ?!? delete[] x; fclose( fp ); 

Answer 1

You use the constructor std::vector std::vector::data to get a pointer to the allocated memory.

Saving with fread :

 std::vector<T> x(big_n); fread(x.data(), sizeof(T), big_n, fp); 

---

As noted by others, using fread if type T not a POD type will most likely not work. Then you can use C ++ and std::istreambuf_iterator to read the file into a vector. However, this has the disadvantage that it big_n over all the elements in the file, and if big_n is as large as it seems, it could be a performance issue.

---

However, if the file is really large, I would rather recommend using memory mapping to read the file.

Answer 2

This will read the file into vector using

 #include <vector> #include <fstream> #include<iterator> // ... std::ifstream testFile("testfile", std::ios::binary); std::vector<unsigned char> fileContents((std::istreambuf_iterator<unsigned char>(testFile)), std::istreambuf_iterator<unsigned char>()); 

This answer comes from a previous answer: stack overflow