Linux default buffer size for file

Question

The documentation states that the default value for buffering is: If omitted, the system default is used . I am currently on Red Hat Linux 6, but I cannot determine the default buffering set for the system.

Can someone explain to me how to determine the buffering for the system?

Answer 1

Since you are associated with 2.7 documents, I assume you are using 2.7. (In Python 3.x, this all becomes much simpler because a lot more buffering is displayed at the Python level.)

Everything open actually (on POSIX systems) is a fopen call, and then, if you passed something for buffering , setvbuf . Since you are not missing anything, you just get the default buffer from fopen , which depends on your standard C library. (For more details see the source . Without buffering it goes from -1 to PyFile_SetBufSize , which does nothing except bufsize >= 0 . )

If you read the glibc setvbuf manpage , this explains that if you never call any buffering functions:

Usually all files are blocked by the buffer. When the first I / O operation occurs in a file, malloc (3) is called and a buffer is obtained.

Note that it does not say which size buffer is received. It is intentional; this means that the implementation can be smart and choose different buffer sizes for different cases. (There is a BUFSIZ constant, but it is only used when calling legacy functions like setbuf ; it is not guaranteed to be used in any other case.)

So what is going on? Well, if you look at the source of glibc, it ultimately calls the _IO_DOALLOCATE macro, which you can plug in (or override, since glibc combines C ++ streambuf and C stdio), but ultimately it highlights buf _IO_BUFSIZE , which is an alias for the specific for the macro platform _G_BUFSIZE , which is 8192 .

Of course, you probably want to trace macros on your own system, rather than trust a generic source.

---

You may wonder why there is no good documented way to get this information. Apparently, this is because you should not care. If you need a specific buffer size, you set it manually; if you believe that the system knows better, just trust it. If you are not really working on a kernel or libc, who cares? Theoretically, this also leaves open the possibility that the system can do something smart here, for example, choosing bufsize based on the file system block size or even based on current statistics, although this does not look like linux / glibc, FreeBSD or OS X is nothing but using a constant. And, most likely, because it really is not important for most applications. (You might want to check this out yourself - use explicit buffer sizes from 1 KB to 2 MB on some buffered I / O-related scripts and see what the performance differences are.)

Answer 2

I'm not sure if this is the correct answer, but the python 3.0 library and python 20 library describe io.DEFAULT_BUFFER_SIZE in the same way as by default in docs for open() . Coincidence?

If not, then for me there was an answer:

 $ python >>> import io >>> io.DEFAULT_BUFFER_SIZE 8192 $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 14.04.1 LTS Release: 14.04 Codename: trusty 

Answer 3

 #include <stdio.h> int main(int argc, char* argv[]){ printf("%d\n", BUFSIZ); return 0; } 

I did "man setvbuf" to find this. setvbuf - footnote [2] on the documentation page.