Geek Answers Handbook

Error when using multiprocessing module in python daemons

I get the following error when using the multiprocessing module in a python daemon process (using python-daemon ):

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func (* targs, ** kargs)
  File "/usr/local/lib/python2.6/multiprocessing/util.py", line 262, in _exit_function
    for p in active_children ():
  File "/usr/local/lib/python2.6/multiprocessing/process.py", line 43, in active_children
    _cleanup ()
  File "/usr/local/lib/python2.6/multiprocessing/process.py", line 53, in _cleanup
    if p._popen.poll () is not None:
  File "/usr/local/lib/python2.6/multiprocessing/forking.py", line 106, in poll
    pid, sts = os.waitpid (self.pid, flag)
OSError: [Errno 10] No child processes

The daemon (parent) process spawns a series of processes (children), and then periodically checks the processes to see if they are complete. If the parent discovers that one of the processes has completed, it then tries to restart the process. It is at this moment that the foregoing is excluded. It seems that as soon as one of the processes is completed, any operation associated with the multiprocessing module will throw this exception. If I run identical code in a non-daemon python script, it runs without any errors.

EDIT:

Script example

from daemon import runner

class DaemonApp(object):
    def __init__(self, pidfile_path, run):
        self.pidfile_path = pidfile_path
        self.run = run

        self.stdin_path = '/dev/null'
        self.stdout_path = '/dev/tty'
        self.stderr_path = '/dev/tty'

def run():
    import multiprocessing as processing
    import time
    import os
    import sys
    import signal

    def func():
        print 'pid: ', os.getpid()
        for i in range(5):
            print i
            time.sleep(1)

    process = processing.Process(target=func)
    process.start()

    while True:
        print 'checking process'
        if not process.is_alive():
            print 'process dead'
            process = processing.Process(target=func)
            process.start()
        time.sleep(1)

# uncomment to run as daemon
app = DaemonApp('/root/bugtest.pid', run)
daemon_runner = runner.DaemonRunner(app)
daemon_runner.do_action()

#uncomment to run as regular script
#run()
+5
source share
6

SIGCLD subprocess - (issue 1731717, 2011-09-21 ).

1.4.8 python-daemon; SIGCLD, .

+4

daemon multiprocessing, , SIGCLD ( ). daemon SIGCLD SIG_IGN , , , Linux, , ( , , wait()). is_alive- wait(), , , , .

- SIGCLD SIG_DFL ( - wait() ):

def run():
    # ...

    signal.signal(signal.SIGCLD, signal.SIG_DFL)

    process = processing.Process(target=func)
    process.start()

    while True:
        # ...
+5

, trunk 2.6 maint, , script python-trunk 2.6-maint svn?

0

, - , ...:

File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)

if atexit._run_exitfuncs , , . , - , multiprocessing "at-exit" . : ? , - : , - OTHER, , , ...

0

celery RHEL 5.3 Python 2.6. , :

      File "/usr/local/lib/python2.6/multiprocessing/pool.py", line 334, in terminate
    self._terminate()
  File "/usr/local/lib/python2.6/multiprocessing/util.py", line 174, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.6/multiprocessing/pool.py", line 373, in _terminate_pool
    p.terminate()
  File "/usr/local/lib/python2.6/multiprocessing/process.py", line 111, in terminate
    self._popen.terminate()
  File "/usr/local/lib/python2.6/multiprocessing/forking.py", line 136, in terminate
    if self.wait(timeout=0.1) is None:
  File "/usr/local/lib/python2.6/multiprocessing/forking.py", line 121, in wait
    res = self.poll()
  File "/usr/local/lib/python2.6/multiprocessing/forking.py", line 106, in poll
    pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 10] No child processes

. pdb, .

0

script " ", . script, , , , . (, ). -, , . (: sleep() , , - )

0

More articles:


All Articles