Redirecting stdout from python script called in * nix not working correctly

Question

I have a Python program in which I have several print statements, followed by calls to some C ++ executables, from which I also get some output in stdout. When I run this python script in a unix terminal, I get the output on the screen, as expected (in the correct order, that is, first from print, and then from C ++ executables). The problem is that I redirect this output to a file like

python test.py > out.txt 

I get output in the wrong order. First I get the output of the C ++ executables, and then the other.

Answer 1

Python uses string buffering when stdout is a tty device, hence the output matches the print order. If the python buffer is redirected, the output of the print statements is output. While C ++ executable output buffering is not handled by python in both cases. Therefore, when the output is redirected, print statements receive buffering and are not output to the file until the buffer is full or the program ends.

sys.stdout.flush() will clear the output of print statements as shown below. Please note that it must follow print statements.

 #!/usr/local/bin/python import os import sys print("Hello1") print("Hello2") sys.stdout.flush() os.system("/python/cppprog.o") print("Bye1") print("Bye2") 

Output:

 ]# python script.py > o.txt ]# cat o.txt Hello1 Hello2 Hello, world! Bye1 Bye2 

Answer 2

You can run python with unbuffered output using the -u command line switch, i.e. you can just call python -u myscript.py and then the output in stdout should be synchronized.

Answer 3

This is because the python stdout buffer and os' stdout buffer are not synchronized.

Try resetting stdout after print statements and before executing C ++ executables.

 import sys sys.stdout.flush()