Handling continuous command output in python

Question

I am new to python using perl for many years. A typical thing that I do all the time: perl opens the command as a pipe and assigns its output to a local variable for processing. In other words:

"open CMD, "$command|";
$output=<CMD>;

piece of cake. I think I can do something similar in python this way:

args=[command, args...]
process=subprocess.Popen(args, stdout=subprocess.PIPE)
output=process.communicate()

so far so good. Now for the big question ...

If I run this command using ssh on multiple platforms, I can then track the descriptors in perl inside the select loop to process the results as they arrive. I found python select and poll modules, but not quite sure how to use them. The documentation says that the file descriptor is accepted in the survey, but when I try to pass the variable "process" above to poll.register (), I get an error that it must be int or have the fileno () method. Since Popen () used stdout, I tried calling

poll.register(process.stdout)

and it no longer throws an error, but instead just hangs.

Any suggestions / pointers on how to do something like this work?

Answer 1

select.poll: fileno ( ):

import os, sys, select, subprocess

args = ['sh', '-c', 'while true; do date; sleep 2; done']
p1 = subprocess.Popen(args, stdout=subprocess.PIPE)
p2 = subprocess.Popen(args, stdout=subprocess.PIPE)

while True:
    rlist, wlist, xlist = select.select([p1.stdout, p2.stdout], [], [])
    for stdout in rlist:
        sys.stdout.write(os.read(stdout.fileno(), 1024))

, , . "" , p1.stdout fileno, . , select.

, stdout os.read stdout.read. , stdout.read(1024) , . EOF, EOF , stdout.read , 1024 .

os.read, , , - , . , 1024 os.read(stdout.fileno(), 1024) stdout.

select.epoll , , "" (FD), os.read, :

import os, sys, select, subprocess

args = ['sh', '-c', 'while true; do date; sleep 2; done']
p1 = subprocess.Popen(args, stdout=subprocess.PIPE)
p2 = subprocess.Popen(args, stdout=subprocess.PIPE)

poll = select.poll()
poll.register(p1.stdout)
poll.register(p2.stdout)

while True:
    rlist = poll.poll()
    for fd, event in rlist:
        sys.stdout.write(os.read(fd, 1024))

FD select.POLLHUP. unregister , , , FD .

, , , , - , , .

Answer 2

import subprocess

p = subprocess.Popen('apt-get autoclean', stdout=subprocess.PIPE, stderr = None, shell=True)

for line in iter(p.stdout.readline, ''):

    print line

p.stdout.flush()
p.stdout.close()

print ("Done")