Why ostringstream is faster than stream

Question

To write many files to a file, I have 2 approaches:

• Write directly with the flow

  ofstream file("c:\\test.txt"); for (int i = 0; i < 10000; ++i) { file << data[i]; } 

• Write to istringstream first, and then write to the stream immediately

   ostringstream strstream; for (int i = 0; i < 10000; ++i) { strstream << data[i]; } ofstream file("c:\\test.txt"); file << strstream.str(); 

Not surprisingly, the second approach is faster; in fact, it is 4 times faster than the first approach on my HP7800.

But why? I know that thestream uses filebuf internally, and ostringstream uses stringbuf as the buffer that they should all be in memory, so it shouldn't make any difference.

What is the difference under the hood?

Answer 1

Do you use std::endl instead of '\n' ? std::endl does two things: it inserts '\n' into the stream, and then flushes the buffer to disk . I saw the code speak of a serious performance hit. (The code was fixed 5-10 times faster than it was fixed.)
Flushing in the line buffer will be much faster than flushing the disk, so this will explain your findings.

If this is not the case, you might consider increasing the size of the buffer:

 const std::size_t buf_size = 32768; char my_buffer[buf_size]; ofstream file("c:\\test.txt"); file.rdbuf()->pubsetbuf(my_buffer, buf_size); for (int i = 0; i < 10000; ++i) { file << data[i]; } 

Answer 2

The drive is slow. Many small records are more expensive than one large.

Answer 3

This may be a problem with the implementation of a particular OS. I also believe that the stream buffer (buflen) is less than 10000, the typical value of which is 4095. So try to run with i <4096, and the response time should be the same!

The reason this happens faster in the second case:

In the first case, when the buffer is full (buflen = 4095bytes), it is written to disk. Thus, for i <10000 this should have been reset 3 times.

While in the second case, all data is first created in RAM, and in one case, it is flushed to the hard drive. Thus, two flushes were saved!