Is there a way to output strings in turn from the currently executing python program?

Question

When the pipeline prints the output using a python script with the grep command, the output from the script is apparently only passed to the next command after the entire script has completed.

For example, in the test_grep.py script, as shown below:

 #!/usr/bin/env python from time import sleep print "message1" sleep(5) print "message2" sleep(5) print "message3" 

when called with ./test_grep.py | grep message ./test_grep.py | grep message will not appear for 10 seconds, and at this time all three lines will appear.

Compare this with the test_grep.sh script:

 #!/usr/bin/env bash echo "message1" sleep 5 echo "message2" sleep 5 echo "message3" 

./test_grep.sh | grep message ./test_grep.sh | grep message will immediately output message1 , and then after 5 seconds at intervals of message2 and message3 .

I expect this to happen only after the python interpreter completes execution, this is the output available for the next command. Is there any way to change this behavior?

Answer 1

You can do it:

• Trying every print in python
• By setting stdout unbuffered
• By setting stdout to line buffering

You can even call python -u to disable buffering.

---

I would choose the line buffering option as it seems the most natural.

 open(file, mode='r', buffering=-1 ....) 

Buffering

- an optional integer used to set the buffering policy. Go to 0 to disable buffering (allowed only in binary mode), 1 to select line buffering (can be used only in text mode) and an integer> 1 to indicate the size of the fixed-size block buffer.

If you do not specify buffering (typical "open"), it will use line buffering if it detects that the output goes directly to TTY, that is, to your screen console. If you connect the output or redirect it to a file, it will switch back to the large (4K / 8K) buffer.

---

How do you set stdout to buffer strings?

You can open stdout again via sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1) .