Having difficulty capturing the output of a subprocess that updates the status in place

Question

So, I'm trying to grab the output of HandBrakeCLI as a Python subprocess. This is not a problem with stderr, as each update creates a new line in the file-like channel. However, with stdout, HandBrakeCLI makes in-place updates, and I'm having difficulty grabbing them. I don’t even assume what are called in-place updates, which makes it difficult to find the appropriate advice.

The only solution I came up with is to write stdout to the actual file and read it, but I would rather do it in a smart way (in memory).

COMMAND = ['HandBrakeCLI', '-v', '-i', 'in.m4v', '-o', 'out.m4v', '-Z', 'Normal'] outfile = open('output.txt', 'w') proc = subprocess.Popen(COMMAND, stdout=outfile, stderr=subprocess.PIPE) infile = open('output.txt', 'r') while proc.poll() is None: print infile.read() infile.seek(0) 

It works, but there should be a better way. When I try to use a connection () or just a simple proc.stdout.read (), I get nothing.

What am I doing wrong? Thanks!

Update

In the @wim view, which I checked to see what the original output provided by HandBrakeCLI, and it looked something like this:

 \rEncoding: task 1 of 1, 0.15 % 

What is the best way to handle stdout with the \ r prefix?

Answer 1

I think the comment I made above about using universal_newlines = True will work.

Here's a sample in-place script called "inplace_output.py"

 import sys import time def main(): for k in range(5): print "{0:03d}\r".format(k), sys.stdout.flush() time.sleep(1) if __name__ == "__main__": main() 

You can run this and see how it writes 000, then 001, etc., each time overwriting the previous output.

Here is a script that runs the above as a subprocess and reads its output line by line:

 import subprocess cmd = ['python', 'inplace_output.py'] proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) while True: out = proc.stdout.readline() print repr(out) if len(out) == 0: break 

If you want to immediately collect all the output when the subprocess program terminates, you can replace the while loop with, say,

 out, err = proc.communicate() lines = out.splitlines() 

Do any of you work for you?