Python subprocess check_output is much slower than calling

Question

I tried to understand why this is happening. I call the command to reboot the network on an Ubuntu 12.04 server.

Fast execution

When I invoke a command using one of the following three methods, it takes about 0.1 second to execute:

• directly in the terminal
• python script using os.system
• python script using subprocess.call

terminal session:

 root@ubuntu :~# time /etc/init.d/networking restart * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m0.105s root@ubuntu :~# time python -c "import os; > os.system('/etc/init.d/networking restart')" * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m0.111s root@ubuntu :~# time python -c "import subprocess; > subprocess.call(['/etc/init.d/networking', 'restart'])" * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m0.111s 

Slow execution

However, if I use subprocess.check_output or Popen and try to read the output, it takes 23 seconds. The path is slower. This seems to be a dramatic difference only when I try to use a function that will return the output commands. I would like to understand why this is happening, and find a solution to execute this command and get its output without taking so much time.

terminal session:

 root@ubuntu :~# time python -c "import subprocess; > print subprocess.check_output(['/etc/init.d/networking', 'restart'])" * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m23.201s root@ubuntu :~# time python -c "from subprocess import Popen, PIPE; > print Popen(['/etc/init.d/networking', 'restart'], stdout=PIPE).stdout.read()" * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m23.201s 

Update

One of the comments suggested trying the tee team. Results that are very interesting. In a terminal without python, if tee is used, it takes the same 23 seconds. I'm still wondering why, but at least it can provide more information about what is going on.

 root@ubuntu :~# time /etc/init.d/networking restart | tee out.txt * Running /etc/init.d/networking restart * Reconfiguring network interfaces... real 0m23.181s 

Answer 1

The code below is based on a wonderful comment by J.F. Sebastian. Below is the code in 0.1 seconds, as expected, and returns the command output to a string.

 from subprocess import check_call, STDOUT from tempfile import NamedTemporaryFile with NamedTemporaryFile() as f: check_call(['/etc/init.d/networking', 'restart'], stdout=f, stderr=STDOUT) f.seek(0) output = f.read()