Opening a file in add mode and search

Question

I came across this strange behavior regarding opening a file in append mode and then trying to find the beginning of the file.

The code should be clear: in the second open, I expect that I can write a line at the beginning of the file, and then f.tell() return 5 (the number of bytes written at the beginning of the file).

The fact is that in Python 2.6.6 and 2.7.6 the final statement works, but, surprisingly, it works in Python 3.3.2.

 # Create new file, write after the 100th byte. f = open("test", "w+b") f.seek(100, 0) f.write(b"end") assert f.tell() == 100 + len("end") f.close() # Open file for writing without overwriting, attempt to write at start of file # of the file, but doesn't. f = open("test", "a+b") f.seek(0, 0) f.write(b"start") assert f.tell() == len("start") f.close() 

So, I made a C program that does the equivalent. It really behaves like Python 2.x versions:

 #include <stdio.h> int main() { FILE *f = fopen("tt", "w+b"); fseek(f, 100, 0); fwrite("x", 1, 1, f); fclose(f); f = fopen("tt", "a+b"); fseek(f, 0, 0); fwrite("y", 1, 1, f); printf("%ld\n", ftell(f)); fclose(f); return 0; } 

This prints 102 , and I consider it canonical (I also looked at the outputs strace -eopen,close,lseek , but I'm not smarter).

So my question is: what kind of embarrassing basic knowledge do I not have?

And why is Python 3 behaving differently?

By the way, I'm on Linux.

Answer 1

Behavior is one action of your operating system.

The Python 2 call open() actually opens the files just like your C code, with a bit more packaging. Both will ask the OS to open the file in the specified mode. Then your OS clearly limits the place where you can search; You are not allowed to read or write data that was there before the file was opened.

In Python 3, I / O has been redesigned, and the new file processing code does not directly go into OS mode, which allows you to go beyond this limitation. You can do the same in Python 2 using the io.open() function.

The documentation of the open() functions certainly warns you of certain OS behavior when you cannot use seek when using a :

[...] 'a' to add (which on some Unix systems means that all entries are added to the end of the file regardless of the current search position).

On Linux, the open() man page only states this behavior:

O_APPEND
The file opens in upload mode. Before each write(2) the file offset is located at the end of the file, as if with lseek(2) .

eg. when you try to write, the file position moves to the end of the file; you are only allowed to add.

I am looking for you a list of OSs that behave this way; Microsoft Windows seems to be doing something similar. The .NET FileMode documentation enumeration states:

Append
[...] Trying to find the position to the end of the file throws an IOException , and any attempt to read the failure throws a NotSupportedException .

In any case, if you want to both write and read from a file (rather than just adding), but don’t trim the file when you open it, use the 'r+' mode. This opens the file for reading and writing. You can add, pushing the end, but at the same time you can search for any point in the file.