How to invalidate file system cache?

Question

I want to measure / optimize the boot performance of a cold boot application, and this is difficult to do without a real reboot, which is obviously not an ideal solution.

Is there a way that I could invalidate the entire cache of the system file so that the associated page calls actually cause disk access so that I can measure the time that the program starts to run?

Information:

I really need the FSCTL_DISMOUNT_VOLUME functionality, but for the system volume.

Answer 1

At least in Windows 7, it seems that trying to open a volume descriptor without FILE_SHARE_WRITE access permissions causes the file system cache to be invalid even if the creation fails.

So I made a program that just calls CreateFile for this purpose.

Basic encoding of the program *:

 TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1vZGUuDQ0KJAAAAAAAAAABdRXZRRR7ikUUe4pFFHuKTGz/ikQUe4pMbOqKRBR7ilJpY2hFFHuKAAAAAAAAAABQRQAATAECAMEgRlkAAAAAAAAAAOAAIwELAQkAAAgAAAACAAAAAAAAoBIAAAAQAAAAIAAAAABAAAAQAAAAAgAABQAAAAAAAAAFAAAAAAAAAAAwAAAAAgAAlr4AAAMAAIQAABAAAAAQAAAAEAAAEAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAMABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC50ZXh0AAAAeQYAAAAQAAAACAAAAAIAAAAAAAAAAAAAAAAAACAAAGAucnNyYwAAAMABAAAAIAAAAAIAAAAKAAAAAAAAAAAAAAAAAABAAABAAAAAAAAAAABVi+yLTQiLQTyLVQyD7BBTVot0CHgz2wPxVzvTdBc5XRB1EoldEDgadAv/RRCLRRA4HAJ19YtGGEiJXfiJRfQ703QNi0X0K0X40egDRfjrA4tFEIt+GDvHcgyL2PfTO99yBDP/6xWL+PfXO8dzAov4i14gjTy7izwPA/mF0nRvg2X8AIl98ClV8ItN/ItdEAPKi1XwihQKOV38dASKCesCMskPvtkPvsory3UUhNJ0O4tN/DtNEHQz/0X8i1UM68mLVfg7VfR0TjPbO8t+BkiJRfTrBECJRfiLRfQ5Rfh3NotVDItNCOlU////i00Ii1YYO8JzF4tWJI0EQg+3BAiLVhyNBIKLBAgDwesM99A7whvAI8frAjPAX15bycNVi+yD7ByLVQhkoTAAAABTM8lWV2aJTeSF0nQVZjkKdBBmg0XkAkEPt/FmgzxyAHXwi3gMg8cMiw+JTfA7z3UKM8BfXlvJw4tVCIXSD4SRAAAAD7dBLGY5ReRzBg+3ReTrAw+3wINl+ABm0egz0maJRf5mO9BzXItxMA+3RfgDwA+3DAaJTeyLTQgPtwQIiUX0M9KNTfSF0nUDjU3sD7cBjVi/ZoP7GXcDg8AgD7fAQmaJAYP6AnXbi0XsZjtF9HI1dzP/Rfhmi0X4ZjtF/nKqi03wD7dBLGY5ReR3HBvA99jrBzsPdQeDyP+FwHULi0EY6Uz///+LTfCLCYlN8DvPD4VB////6TX///9Vi+yD7BhTag6NRehQ/3UIM9vGRehfxkXpX8ZF6nfGRetnxkXsZcZF7XTGRe5txkXvYcZF8GnGRfFuxkXyYcZF83LGRfRnxkX1c4hd9uiG/f//g8QMO8N0Eo1N+FFTVo1N/FdR/9CDxBTrB4kfiV38iR6LRfxbycNVjWwkiIHsoAACAFNWV2ptWGpzZolF5FhqdmaJReZYamNmiUXoWGpyZolF6lhqdGaJRexYai5miUXuWGpkZolF8FhqbFtmiUXyi8Nqa2aJRfRmiUX2WMZFdF/GRXVpxkV2b8ZFd2LGRWhmxkVpd8ZFanDGRWtyxkVsacZFbW7GRW50xkVvZsZFNEfGRTVlxkU2dMZFN0zGRThhxkU5c8ZFOnTGRTtFxkU8csZFPXLGRT5vxkU/csZFQEOIXUHGRUJvxkVDc8ZFRGXGRUVIxkVGYcZFR27GRUhkiF1JxkVKZcZFTEPGRU1yxkVOZcZFT2HGRVB0xkVRZcZFUkbGRVNpiF1UxkVVZcZFVldmiUUQamVYanJmiUUSWGpuZolFFFhqZWaJRRZYZolFGGozi8NmiUUaWGaJRRxqMlhmiUUeai5YZolFIGpkWGaJRSKLw2aJRSRmiUUmM8BmiUUojUUQUOgX/f//iUVwM8CNfTCrjUXkahSJRTBYM/ZWiXVkZolFLGaJRS7o8vz//2oKjU1YUVDGRVhMxkVZZMZFWnLGRVtMxkVcb8ZFXWHGRV5kxkVfRIhdYIhdYeij+///g8QUjU1kUY1NLFFWVv/Q/3VkjXXYjX386K79//+JRfhqCI1FaFD/dWTodfv//4lFbGoMjUU0UP91cOhk+///iUUwaguNRUBQ/3Vw6FP7//+JRdxqC41FTFD/dXDoQvv//4lF4GoEjUV0UP91ZOgx+///g2VwAIvwM8BAg8RAg8ZAOUX4iUV0D46EAQAAM8k5TXAPhXkBAACLVXSLRfyLBJAPtxBqQV9mO/p3SmaD+lp3RGaDeAI6dT0Pt1AEZoXSdAxmg/pcdS5mOUgGdShqXFlqLmaJjdj//f9miY3a//3/WWpcZomN3P/9/1lqBGaJjd7//f9Zi9APtwAz/2Y7x3QWD7fAZomETdj//f9BQkIPtwJmO8d17VdXagNXM8BqAWaJhE3Y//3/agGNhdj//f9Q/1Xgg/j/dAlQ/1Xc6b8AAAD/VTBqIIvQWYlVcDvRD4SsAAAAg/oFD4SjAAAAakVYanJmiUUAWGpvi/hmiX0CZol9BF9qJWaJfQaL+GaJfQiL+WaJfQpfamRmiX0MX2pjZol9Dov5Zol9EF9qZWaJfRKL+2aJfRRfamFmiUUaZol9Fl9qaVhqbmaJRRxYamdmiUUeWGolZolFIFhqc2aJRSRYZolFJmoKWGaJRSgzwGaJRSqLRfxmiU0ii010Zol9GP80iI1FAFJQVv9VbIPEEP9FdItFdDtF+A+MfP7//+DQogIDx0cnVzdEluZm8geG1sbnM9InVybjpzY2hlbWFzLW1pY3Jvc29mdC1jb206YXNtLnYzIj4NCiAgICA8c2VjdXJpdHk+DQogICAgICA8cmVxdWVzdGVkUHJpdmlsZWdlcz4NCiAgICAgICAgPHJlcXVlc3RlZEV4ZWN1dGlvbkxldmVsIGxldmVsPSJyZXF1aXJlQWRtaW5pc3RyYXRvciIgdWlBY2Nlc3M9ImZhbHNlIj48L3JlcXVlc3RlZEV4ZWN1dGlvbkxldmVsPg0KICAgICAgPC9yZXF1ZXN0ZWRQcml2aWxlZ2VzPg0KICAgIDwvc2VjdXJpdHk+DQogIDwvdHJ1c3RJbmZvPg0KPC9hc3NlbWJseT5QQURQQURESU5HWFhQQURESU5HUEFERElOR1hYUEFERElOR1BBRERJTkdYWFBBRERJTkdQQURESU5HWFhQQURESU5H 

Source:

 // Usage: ClearCache C: D: #include <tchar.h> #include <stdio.h> #include <windows.h> int _tmain(int argc, LPTSTR argv[]) { LPCTSTR DOS_PREFIX = _T("\\\\.\\"); for (int i = 1; i < argc; i++) { LPTSTR arg = argv[i]; LPTSTR path = (LPTSTR)calloc( _tcslen(arg) + _tcslen(DOS_PREFIX) + 1, sizeof(*arg)); __try { if (_istalpha(arg[0]) && arg[1] == _T(':') && (arg[2] == _T('\0') || arg[2] == _T('\\') && arg[3] == _T('\0'))) { _tcscat(path, DOS_PREFIX); } _tcscat(path, arg); HANDLE hFile = CreateFile(path, FILE_READ_DATA, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL); if (hFile != INVALID_HANDLE_VALUE) { CloseHandle(hFile); } else { DWORD le = GetLastError(); if (le != ERROR_SHARING_VIOLATION && le != ERROR_ACCESS_DENIED) { _ftprintf(stderr, _T("Error %d clearing %s\n"), le, argv[i]); return le; } } } __finally { free(path); } } return 0; } 

_{* Just for fun, see if you can understand what the executable does by parsing it. This is not your typical executable. :) Sub>}

Answer 2

I wrote a simple command line utility for this: FlushFileCache

It relies on the undocumented functions of NtSetSystemInformation and can also flush various memory pools.

Answer 3

What did David say. Create a large file, however you need a lot of GB, and each time you want to reset your file cache, make a copy of the file. Then make sure you delete the old file.

So, create BIGFILE1.DAT, copy it to BIGFILE2.DAT, and then delete BIGFILE1.DAT (which removes it from disk and cache). Next time just change the process.

Addenda:

Well, another option is to take the files that are mapped and copy them to new files, delete the old ones and rename the new files back to the old ones. The cache is supported by the file. If the file "leaves", then the cache.

If you can identify these files and they are not used by the system / other running programs, this should be simple for the script and, ideally, work faster than copying 6G files.

Answer 4

You can use the virtual machine and make a shotshot immediately after loading the virtual machine. Resuming from a snapshot will be faster than rebooting.

Answer 5

This solution worked perfectly: stack overflow

More specifically, I am doing this:

 // open with FILE_FLAG_NO_BUFFERING auto hFile = CreateFile(path.c_str(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_FLAG_NO_BUFFERING, nullptr); /// check if (hFile == INVALID_HANDLE_VALUE){ //_tprintf(TEXT("Terminal failure: unable to open file \"%s\" for read.\n"), argv[1]); cout << "error" << endl; return; } // close CloseHandle(hFile); // now use regular C++ File API without caching: ifstream file(path, ios::binary | ios::ate);