Python shell wrapper

Question

I am trying to write a simple version of the time command-line program in Python, except that instead of mapping real / usr / sys time to the shell, it writes it to the database.

I currently have:

wrapper.py

 #!/usr/bin/python import sys from subprocess import Popen, PIPE cmd = 'time ' + (' '.join(['"%s"' % v if ' ' in v else v for v in sys.argv[1:]])) p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) print p.stdout.read() print p.stderr.read() 

For simplicity, I excluded the database insert code.

However, to illustrate the problem, I use a test script:

 #!/usr/bin/python import time for i in xrange(3): print i time.sleep(1) 

If I ran wrapper.py python delay.py , I would like the seconds to be displayed in real time, and then something like:

 real 0m3.057s user 0m0.030s sys 0m0.000s 

Instead, I don't get output for 3 seconds, and then it prints:

 0 1 2 0.02user 0.00system 0:03.03elapsed 0%CPU (0avgtext+0avgdata 21632maxresident)k 0inputs+0outputs (0major+1514minor)pagefaults 0swaps 

How do you read and print output from a subprocess, how does this happen in real time?

Also, why is the output from time different than when I run it directly in the shell and get messed up when running from a subprocess inside a Python script?

Answer 1

First, why are you working with I / O with python? Just let the stdout and stderr sub process go to the same place as python. Secondly, instead of actually using a time command, you can extract resources directly from python. Try something like this:

 #! /usr/bin/python import os import resource import sys import time cmd = ' '.join(sys.argv[1:]) stime = time.time() os.system(cmd) # fire off the command etime = time.time() # now get the resource utilization r = resource.getrusage(resource.RUSAGE_CHILDREN) user_time = r.ru_utime sys_time = r.ru_stime # resource doesn't know "clock" time, so we'll just figure that ourselves real_time = etime - stime print "real\t" + str(real_time) print "user\t" + str(user_time) print "sys\t" + str(sys_time) 

Print time in seconds. If you really want them to look just like a time command, you can format them accordingly.

To answer the second part of your question, there are actually different time commands. When you run it as a child of python, you get the output of / usr / bin / time. When you start it manually, you get a built-in version of the shell time. Try typing "type -a time" on the command line. Also, try running the test program as follows: "/usr/bin/time./test.py" and you should see the second form.

Answer 2

You need to record the output in real time:

 process = subprocess.Popen(shlex.split(command), stdout = subprocess.PIPE, stderr = subprocess.STDOUT) while True: output = process.stdout.read(1) if output == '' and process.poll() != None: break if output != '': sys.stdout.write(output) sys.stdout.flush()