Opening an already open file does not raise an exception

Question

Consider these two python programs:

script_a.py :

 from datetime import datetime from time import sleep while True: sleep(1) with open('foo.txt', 'w') as f: sleep(3) s = str(datetime.now()) f.write(s) sleep(3) 

script_b.py :

 while True: with open('foo.txt') as f: s = f.read() print s 

Run script_a.py . While it is running, run script_b.py . Both will succeed, but script_b.py displays an empty string if the file is currently open by script_a.py .

I was expecting an IOError exception IOError be raised, saying that the file is already open, but this did not happen, instead the file looks empty. Why is this and what would be the right way to check if it is open by another process? Would it be ok to just check if an empty string is returned and try again until something else is read, or is there a more pythonic way?

Answer 1

See another answer and comments on how many files open in Python. If you read all this and want to block access to the file on the POSIX platform, you can use the fcntl library.

Keep in mind that: A) other programs can ignore your lock in the file, B) some network file systems do not lock very well or generally C) be very careful to release locks and avoid deadlock as flock will not detect it [1 ] [2] .

Example .... script_a.py

 from datetime import datetime from time import sleep import fcntl while True: sleep(1) with open('foo.txt', 'w') as f: s = str(datetime.now()) print datetime.now(), "Waiting for lock" fcntl.flock(f, fcntl.LOCK_EX) print datetime.now(), "Lock clear, writing" sleep(3) f.write(s) print datetime.now(), "releasing lock" fcntl.flock(f, fcntl.LOCK_UN) 

script_b.py

 import fcntl from datetime import datetime while True: with open('foo.txt') as f: print datetime.now(), "Getting lock" fcntl.flock(f, fcntl.LOCK_EX) print datetime.now(), "Got lock, reading file" s = f.read() print datetime.now(), "Read file, releasing lock" fcntl.flock(f, fcntl.LOCK_UN) print s 

Hope this helps!

Answer 2

You are allowed to open the file as many times as you like if the operating system does not stop you. It is sometimes useful to get multiple cursors per file for complex operations.

The reason script_b.py thinks the file is empty is because the file empty:

 with open('foo.txt', 'w') as f: 

opening a file in w mode immediately erases (i.e. truncates) the file. There, the initial three second space in script_a , where the file is completely 100% empty, and what script_b sees.

In the next three second interval after calling f.write file is still ... maybe empty. This is due to buffering - the file on the disk is not guaranteed to contain everything that you wrote to it using write until it is close (i.e., exit the context manager block) or manually call flush in the file descriptor.

In addition, you can open in unbuffered mode, so that recordings are always written to disk.

 with open('foo.txt','w',0) as f: #no buffering, f.write() writes immediately to disk