Geek Answers Handbook

Forcing a file to disk

I am currently implementing a ping pong buffering scheme to safely write a file to disk. I am using C ++ / Boost on a Linux / CentOS machine. Now I am facing a problem to force a file to be written to disk. Is it possible to do this regardless of all caching policies for custom file system rules (ext3 / ext4) / SO / RAID controller / hard disk controller?

Is it better to use the usual functions fread () / fwrite (), C ++ or to increase the file system?

I heard that just flushing the file (fflush ()) does not guarantee the actual recording

+2
source share
3 answers

fflush (for FILE *), std :: flush (for IOStream) to force your program to send to the OS.

Posix has

  • sync (2) ask to schedule recording its buffers, but may return before writing (Linux expects data to be sent to the hardware before returning).

  • fsync (2), which is guaranteed to wait for data to be sent to the hardware, but it needs a file descriptor (you can get it from FILE * with fileno (3), I don’t know the standard way to get one from IOStream).

  • O_SYNC as a flag to open (2).

In all cases, the hardware can have its own buffers (but if it has control over it, a good implementation will also try to clear them and ISTR that some drives use capacitors so that they can flush everything that happens with the power) and network file systems have their own reservations.

+5
source

You can use fsync () / fdatasync () to force (Note 1) data to be stored. They require a file descriptor, as indicated, for example, by open (). The linux manpage has more information about a particular Linux, especially about the differences between fsync and fdatasync.

If you do not use file descriptors directly, many abstractions will contain internal buffers that are in your process.

eg. if you are using FILE *, you first need to unload the data from your application.

//... open and write data to a FILE *myfile fflush(myfile); fsync(fileno(myfile));
  • Note 1: These calls force the OS to ensure that any data in any OS cache is written to the disk, and the disk confirms this. Many hard drives relate to the OS about this and can write data to the cache on the disk.
+1
source

Not in standard C ++. You will have to use some specific IO system, for example open with the O_SYNC flag under Unix, and then write .

Note that this is partially implied by the fact that ostream (and in C, FILE* ) are buffered. If you do not know exactly when something is written to disk, then there is no point in insisting on the transactional integrity of the record. (It would not be too difficult to design a streambuf , which is written only with an explicit flash, however.)

EDIT:

As a simple example:

 class SynchronizedStreambuf : public std::streambuf { int myFd; std::vector<char> myBuffer; protected: virtual int overflow( int ch ); virtual int sync(); public: SynchronizedStreambuf( std::string const& filename ); ~SynchronizedStreambuf(); }; int SynchronizedStreambuf::overflow( int ch ) { if ( myFd == -1 ) { return traits_type::eof(); } else if ( ch == traits_type::eof() ) { return sync() == -1 ? traits_type::eof() : 0; } else { myBuffer.push_back( ch ); size_t nextPos = myBuffer.size(); myBuffer.resize( 1000 ); setp( &myBuffer[0] + nextPos, &myBuffer[0] + myBuffer.size() ); return ch; } } int SynchronizedStreambuf::sync() { size_t toWrite = pptr() - &myBuffer[0]; int result = (toWrite == 0 || write( myFd, &myBuffer[0], toWrite ) == toWrite ? 0 : -1); if ( result == -1 ) { close( myFd ); setp( NULL, NULL ); myFd = -1; } else { setp( &myBuffer[0], &myBuffer[0] + myBuffer.size() ); } return result; } SynchronizedStreambuf::SynchronizedStreambuf( std::string const& filename ) : myFd( open( filename.c_str(), O_WRONLY | O_CREAT | O_SYNC, 0664 ) ) { } SynchronizedStreambuf::~SynchronizedStreambuf() { sync(); close( myFd ); }

(This has been verified only superficially, but there is a basic idea.)

0
source

More articles:


All Articles