What does SetFileValidData do? What is the difference with SetEndOfFile?

Question

I am looking for a way to expand a file asynchronously and efficiently.

In the support document Asynchronous Data Input / Output Synchronously in Windows NT, Windows 2000, and Windows XP :

NOTE. Applications can perform the aforementioned write operation asynchronously, changing the permissible length of the file data using the SetFileValidData function, and then issuing WriteFile.

on MSDN, SetFileValidData is a function for Sets the valid data length of the specified file .

But I still do not understand what "reliable data" is, what is the difference between it and file size?

I can use SetFilePointerEx and SetEndOfFile to expand the file size, but how to do it with SetFileValidData ?

SetFileValidData cannot enter an argument other than file size. In this case, what is the lively meaning of SetFileValidData ?

Answer 1

When you use SetEndOfFile to increase the length of the file, the length of the logical file changes and the necessary disk space is allocated, but in fact the data is not physically written to the disk sectors corresponding to the new part of the file. The actual data length remains the same as it.

This means that you can use SetEndOfFile to make the file very large very quickly, and if you read from the new part of the file, you will only get zeros. The permissible data length increases when you write the actual data to a new part of the file.

This is great if you just want to reserve a place and then write data to a file sequentially. But if you make the file very large and immediately write the data closer to the end, zeros should be written to a new part of the file, which will take a lot of time. If you really don't need a file to store zeros, you can use SetFileValidData to skip this step; the new part of the file will contain random data from previously deleted files.

Answer 2

Note that setendoffile does not write any zeros to any selected sectors on the disk, it just highlights the pointers to spaces inside the MFT records and then updates the bitmap image of the entire file system space. But OS or FS write the valid / logical length of the file in the MFT record, if you increase it from 1 to 2 GB, then the attached 1 GB must be all zeros, but FS will not write zeros to disks, refer to this file with a valid length, to know that 1 GB should be zeros, if you try to read access to this enlarged part of 1 GB, it will fill in the Zeers directly in RAM, and then feedback with your application. But if you write access to any byte inside this part of 1 GB, FS should fill with zeros from the initial 1GB offset to the current pointer that your application is trying to apply, but not other bytes from the current location to the tail of the file, but meanwhile fixes a valid / logical lengths from 0 to current location, physical size and allocated size are still 2 GB. But if you use setfilevaliddata, FS will set the allowable length to 2GB directly and won't bother to fill in any zeros, no matter where you write, just write, but anywhere you read, you can read some garbage data that was created by others applications.

Answer 3

Agree with Harry Johnston's answer from a practical point of view, while SetFileValidData has a performance advantage because it does not require written zeros, it has security implications because the file may contain data from other deleted files. Therefore, the special privilege SE_MANAGE_VOLUME_NAME is required, as mentioned on MSDN: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365544(v=vs.85).aspx

The reason is that if the user account of the running program does not have this privilege, then using SetFileValidData can map other data deleted by the user into the view of this particular file, so ordinary users (non-administrators) are not allowed to do this. Even for privileged users, they still need to make sure to use ACLs (access control lists) in the file system to protect this file so that it is not shared with users with unprivileged users.

Answer 4

It seems that SenEndofFile does not actually allocate the reserved disk space for the target file, SetFileValidData is responsible for the task.

MSDN Link ,

You can use the SetFileValidData function to create large files in special circumstances so that the performance of subsequent file I / O is better than others. In particular, if the extended part of the file is large and will be written in random order, for example, in the type of database application, the time required to extend and write to the file will be faster than using SetEndOfFile and writing is arbitrary.

If SetEndOfFile really sets a space, then SetFileValidData will not do anything better than SetEndOfFile when writing. Therefore, SetEndOfFile can simply create a sparse file with holes, and SetFileValidData is the actual distribution.