Why does my Python3 script overlap the output of its output to the head or tail (sys module)?

Question

I have a Python3 script that writes its output to stdout, but it complains when I process this output to the head or tail. Note that the example below shows that this works, since head returns the requested first two lines of output.

 > ./script.py '../Testdata/*indels.ss' -m 5 | head -2 ~/Databases/Avian_genomes/Sandbox/combined xread 2999 50 Traceback (most recent call last): File "./new.py", line 194, in <module> sys.stdout.write(lineout) IOError: [Errno 32] Broken pipe Exception IOError: IOError(32, 'Broken pipe') in <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'> ignored 

In contrast, the script has no problems with its output connected to awk, as shown below.

 > ./script.py '../Testdata/*indels.ss' -m 5 | awk 'NR < 3 {print $0}' ~/Databases/Avian_genomes/Sandbox/combined xread 2999 50 

Let me know if you need the code from the script other than what the error message contains. I'm not sure what will be relevant.

Answer 1

 ./script.py '../Testdata/*indels.ss' -m 5 | awk 'NR >= 3 {exit} 1' 

will show the same behavior as head -2 .

You can configure the SIGPIPE handler to one that quietly kills your program:

 import signal signal.signal(signal.SIGPIPE, signal.SIG_DFL) 

Answer 2

I will quote from here :

If a sequence of commands appears in the pipeline, and one of

read commands end before recording is completed,

receives a SIGPIPE signal.

What does the head do. Your script has not finished writing, but head already executed, so stdout closed, so an exception.