Python: a simple approach to killing children or reporting their success?

Question

I want to

• call shell commands (for example, "sleep" below) in parallel,
• report on their individual launches and completion and
• you can kill them with "kill -9 parent_process_pid".

Much has already been written about these things, but I feel that I have not quite found the elegant pythonic solution that I am looking for. I also try to keep things relatively readable (and short) for someone completely unfamiliar with python.

My approach so far (see code below):

• put subprocess.call (unix_command) in a wrapper function that reports the start and end of the command.
• calling a wrapper function using multiprocess.Process.
• track corresponding pids, store them all over the world and kill them in signal_handler.

I tried to avoid a solution periodically checking processes, but I'm not sure why.

Is there a better approach?

import subprocess,multiprocessing,signal
import sys,os,time

def sigterm_handler(signal, frame):
        print 'You killed me!'
        for p in pids:
                os.kill(p,9)
        sys.exit(0)

def sigint_handler(signal, frame):
        print 'You pressed Ctrl+C!'
        sys.exit(0)

signal.signal(signal.SIGINT, sigint_handler)
signal.signal(signal.SIGTERM, sigterm_handler)

def f_wrapper(d):
        print str(d) + " start"
        p=subprocess.call(["sleep","100"])
        pids.append(p.pid)
        print str(d) + " done"

print "Starting to run things."

pids=[]

for i in range(5):
        p=multiprocessing.Process(target=f_wrapper,args=(i,))
        p.daemon=True
        p.start()

print "Got things running ..."

while pids:
        print "Still working ..."
        time.sleep(1)

Answer 1

As soon as it subprocess.callreturns, the subprocess is executed - and the callreturn value is a subprocess returncode. Thus, accumulating these return codes in the list pids(which is not synchronized between multiprocess addition and the main process) and sending them 9signals “as if”, they were process identifiers instead of returning the codes, of course, are erroneous.

Another thing with a question that is definitely incorrect is the specification:

kill -9 parent_process_pid".

-9 , ( -9) - , -9 .

threading multiprocessing ( "babysitter", , , , ​​ ?); suprocess.Process ( .pid ) babysitter, ( ). , , , , , ( ), , .

, ( , ;-) s/thing, :

import subprocess, threading, signal
import sys, time

pobs = set()
pobslock = threading.Lock()
def numpobs():
    with pobslock:
        return len(pobs)

def sigterm_handler(signal, frame):
    print 'You killed me!'
    with pobslock:
        for p in pobs: p.kill()
    sys.exit(0)

def sigint_handler(signal, frame):
    print 'You pressed Ctrl+C!'
    sys.exit(0)

signal.signal(signal.SIGINT, sigint_handler)
signal.signal(signal.SIGTERM, sigterm_handler)

def f_wrapper(d, p):
    print d, 'start', p.pid
    rc = p.wait()
    with pobslock:
        pobs.remove(p)
    print d, 'done, rc =', rc

print "Starting to run things."

for i in range(5):
    p = subprocess.Popen(['sleep', '100'])
    with pobslock:
        pobs.add(p)
    t = threading.Thread(target=f_wrapper, args=(i, p))
    t.daemon=True
    t.start()

print "Got things running ..."

while numpobs():
    print "Still working ..."
    time.sleep(1)

Answer 2

( ), , , "top" ctrl-c . Alex subprocess.Process subprocess.Popen call ( , subprocess.Process ).

stdout , .

import subprocess, threading, signal
import sys, time

pobs = set()                            # set to hold the active-process objects
pobslock = threading.Lock()     # a Lock object to make sure only one at a time can modify pobs

def numpobs():
        with pobslock:
                return len(pobs)

# signal handlers
def sigterm_handler(signal, frame):
        print 'You killed me! I will take care of the children.'
        with pobslock:
                for p in pobs: p.kill()
        sys.exit(0)

def sigint_handler(signal, frame):
        print 'You pressed Ctrl+C! The children will be dealt with automatically.'
        sys.exit(0)

signal.signal(signal.SIGINT, sigint_handler)
signal.signal(signal.SIGTERM, sigterm_handler)


# a function to watch processes
def p_watch(d, p):
        print d, 'start', p.pid
        rc = p.wait()
        with pobslock:
                pobs.remove(p)
        print d, 'done, rc =', rc


# the main code
print "Starting to run things ..."
for i in range(5):
        p = subprocess.Popen(['sleep', '4'])
        with pobslock:
                pobs.add(p)
        # create and start a "daemon" to watch and report the process p.
        t = threading.Thread(target=p_watch, args=(i, p))
        t.daemon=True
        t.start()

print "Got things running ..."
while numpobs():
        print "Still working ..."
        time.sleep(1)