Unable to read Python to end of file

Question

I tried this in several different ways, but the result always seems the same. I can't get Python to read to the end of the file here. He stops about halfway. I tried binary and ASCII modes, but both of them have the same result. I also checked for any special characters in the file where it is disconnected, and there are none. In addition, I tried to indicate how much to read, and it still disconnects in the same place.

It looks something like this:

f=open("archives/archivelog", "r") logtext=f.read() print logtext 

This happens if I call it from bash or from python, regardless of whether I am a regular user or root.

HOWEVER, it works fine if the file is in the same directory as me.

 f=open("archivelog", "r") logtext=f.read() print logtext 

It works like a dream. Any idea why?

Answer 1

Bouncing back from Kelketek's answer:

I cannot remember where I read about this, but basically the Python garbage collector works "from time to time", without any guarantees as to when this object will be collected. The flush() function does the same: http://docs.python.org/library/stdtypes.html#file.flush . I compiled in that flush() puts the data in some buffer for writing, and it depends on your OS to decide when to do it. Probably one or both of them was your problem.

Did you read in the file shortly after writing it? This can lead to a race condition ( http://en.wikipedia.org/wiki/Race_condition ), which is a class of usually strange, possibly random / hard-to-reproduce errors that you would not normally expect from a high-level language such as Python.

Answer 2

The Python reference manual on read() says:

Also note that in non-blocking mode, less data than was requested can be returned, even if the size parameter is not specified.

There is also a PEP project on this subject, which, apparently, was not adopted. PEP is a Python Improvement Suggestion .

So, the sad state of things is that you cannot rely on read() to give you a complete file in one call.

If the file is a text file, I suggest using readlines() instead. It will provide you with a list containing each line of the file. As far as I can tell, readlines() reliable.

Answer 3

The read method returns the contents of the file in chunks. You must call it again until it returns an empty string ('').

http://docs.python.org/tutorial/inputoutput.html#methods-of-file-objects

Answer 4

Ok, write this in a notebook first, so I don't press 'enter' too soon ...

I solved the problem, but I'm not sure why the solution solves this problem.

As it turns out, the reason why you could read, and not another, was that the one that was cut off earlier was created using a Python script, while the other was created earlier.

Despite the fact that I closed the file, the file was not completely written to disk, or when I grabbed it, it was just what was in the buffer. Something like that.

Performing:

  del f 

And then, trying to capture the file, I got the whole file. And yes, I used f.close after writing the file.

So, the problem is solved, but can someone explain to me the reason why I had to manually collect garbage in this case? I did not think that I would need to do this in Python.