Python multiprocessing: crashing in a subprocess?

Question

What happens when a python script opens subprocesses and one process crashes?

stack overflow

Will there be a major process failure?

Failure of other subprocesses?

Is there a signal or other event that is spreading?

Answer 1

When using multiprocessing.Pool, if one of the subprocesses in the pool fails, you will not be notified at all, and the new process will immediately begin to take its place:

>>> import multiprocessing
>>> p = multiprocessing.Pool()
>>> p._processes
4
>>> p._pool
[<Process(PoolWorker-1, started daemon)>, <Process(PoolWorker-2, started daemon)>, <Process(PoolWorker-3, started daemon)>, <Process(PoolWorker-4, started daemon)>]
>>> [proc.pid for proc in p._pool]
[30760, 30761, 30762, 30763]

Then in another window:

dan@dantop:~$ kill 30763

Back to the pool:

>>> [proc.pid for proc in p._pool]
[30760, 30761, 30762, 30767]  # New pid for the last process

, . , , . map apply, , . , ​​ concurrent.futures.ProcessPoolExecutor, multiprocessing.Pool. Python 3.3, ProcessPoolExecutor BrokenProcessPool, , . , multiprocessing . , - , .

. , , , . , :

>>> def f(): raise Exception("Oh no")
... 
>>> pool = multiprocessing.Pool()
>>> result = pool.apply_async(f)
>>> result.get()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 528, in get
    raise self._value
Exception: Oh no

multiprocessing.Process , , :

>>> def f(): time.sleep(30)
... 
>>> p = multiprocessing.Process(target=f)
>>> p.start()
>>> p.join()  # Kill the process while this is blocking, and join immediately ends
>>> p.exitcode
-15

, :

from multiprocessing import Process

def f(x):
    raise Exception("Oh no")

if __name__ == '__main__':
    p = Process(target=f)
    p.start()
    p.join()
    print(p.exitcode)
    print("done")

:

Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.2/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/usr/lib/python3.2/multiprocessing/process.py", line 116, in run
    self._target(*self._args, **self._kwargs)
TypeError: f() takes exactly 1 argument (0 given)
1
done

, , , exitcode 1.