Python: subprocess.call broken pipe

Question

I am trying to call a shell script in python, but it continues to report a damaged channel error (the result is ok, but I don't want to see an error message in STDERR). I have identified the cause, and it can be reproduced as the following snippet:

subprocess.call('cat /dev/zero | head -c 10 | base64', shell=True)

AAAAAAAAAAAAAAA ==

cat: write error: broken pipe

/dev/zero is an infinite stream, but head -c 10 only reads 10 bytes from it and exits, then the cat will receive SIGPIPE due to the peer closing the pipe. There is no pipe break error message when I run a command in the shell, but why does python show it?

Answer 1

The default action for the SIGPIPE signal is to terminate the program. The Python interpreter changes it to SIG_IGN to be able to report broken pipe errors to the program as exceptions.

When you execute cat ... |head ... in a shell, cat has a default SIGPIPE handler, and the kernel simply terminates it on SIGPIPE.

When you execute cat using subprocess , it outputs the SIGPIPE handler from its parent (python interpreter), SIGPIPE is simply ignored, and cat handles the error itself by checking the errno variable and print error message. A.

To avoid error messages from cat , you can use the preexec_fn argument for subprocess.call:

 from signal import signal, SIGPIPE, SIG_DFL subprocess.call( 'cat /dev/zero | head -c 10 | base64', shell = True, preexec_fn = lambda: signal(SIGPIPE, SIG_DFL) ) 

Answer 2

In this trivial case, at least you don't type anything using the mdash shell commands and lose portability and speed.

Python 2 code:

 >>> import base64 >>> base64.b64encode(open('/dev/zero', 'rb').read(10)) 'AAAAAAAAAAAAAA==' >>> base64.b64encode('\0' * 10) 'AAAAAAAAAAAAAA==' 

In Python 3 (the code will also work in version 2.6+, although it will return instances of str , not bytes ):

 >>> import base64 >>> base64.b64encode(open('/dev/zero', 'rb').read(10)) b'AAAAAAAAAAAAAA==' >>> base64.b64encode(b'\0' * 10) b'AAAAAAAAAAAAAA==' 

In each case, the first example retains the use of /dev/zero (which is not portable in itself, but it doesn't matter), the second produces an effect, although I assume that this is not what you want specifically?