Why multiprocessing. Is a process capable of revealing decorated functions?

Question

So, I read here that the decorated functions cannot be pickled. Really:

import multiprocessing as mp

def deco(f):
    def wrapper(*args, **kwargs):
        try:
            f(*args, **kwargs)
        except:
            print 'Exception caught!'
    return wrapper

@deco
def f(x):
    print x
    raise OverflowError 

if __name__ == '__main__':
    pool = mp.Pool(processes=1)
    for _ in pool.imap_unordered(f, range(10)):
        pass
    pool.close()
    pool.join()
    print 'All done'

Of:

Traceback (most recent call last):
  File "deco0.py", line 19, in <module>
    for _ in pool.imap_unordered(f, range(10)):
  File "/Users/usualme/anaconda/lib/python2.7/multiprocessing/pool.py", line 659, in next
    raise value
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

But now, if I replaced mapwith a Process:

import multiprocessing as mp

def deco(f):
    def wrapper(*args, **kwargs):
        try:
            f(*args, **kwargs)
        except:
            print 'Exception caught!'
    return wrapper

@deco
def f(x):
    print x
    raise OverflowError 

if __name__ == '__main__':
    p = mp.Process(target=f, args=(1,))
    p.start()
    p.join()
    print 'All done'

Of:

1
Exception caught!
All done

Why does it work? Is it also necessary to handle the decorated function?

Answer 1

, Linux, f, Process.__init__. , f os.fork. Windows ( fork), f Pool.apply/Pool/map ( f, ) .

, :

import multiprocessing as mp

def deco(f):
    def wrapper(*args, **kwargs):
        try:
            f(*args, **kwargs)
        except:
            print 'Exception caught!'
    return wrapper

@deco
def f(x):
    print x
    raise OverflowError 

if __name__ == '__main__':
    p = mp.Pool()
    p.apply(f, args=(1,))  # f needs to be pickled here.
    print 'All done'

:

1
Exception caught!
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 319, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed