Geek Answers Handbook

Python Pickle throws FileNotFoundError

I have a situation in python where in a python-launched object that works with other processes, the following situation arises: If the code is simple:

f = open(filename, "rb")
f.close()

There is no error, but then, when the code changes to the following, entering the pickle in the middle, it throws FileNotFoundError:

f = open(filename, "rb")
object = pickle.load(f)
f.close()

I do not understand why, if the file exists, pickling throws such an error. Full error trace:

task = pickle.load(f)
File "/usr/lib/python3.4/multiprocessing/managers.py", line 852, in RebuildProxy
   return func(token, serializer, incref=incref, **kwds)
File "/usr/lib/python3.4/multiprocessing/managers.py", line 706, in __init__
   self._incref()
File "/usr/lib/python3.4/multiprocessing/managers.py", line 756, in _incref
   conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python3.4/multiprocessing/connection.py", line 495, in Client
   c = SocketClient(address)
File "/usr/lib/python3.4/multiprocessing/connection.py", line 624, in SocketClient
   s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
+4
source share
1 answer

Although I would say that @TesselatingHeckler provided the answer, this comment really does not fit in the comments section ... so this is an extension of this answer.

multiprocessing Queue Pipe , , load.

multiprocessing, . , pickle (dill). pathos ( github). Pipes , , python socket python Queue. - multiprocessing Queue.

>>> from processing import Pipe
>>> p = Pipe()
>>> 
>>> import dill
>>> dill.loads(dill.dumps(p))
(Connection(handle=12), Connection(handle=14))
>>>
>>> from socket import socket
>>> s = socket()
>>> from Queue import Queue as que
>>> w = que()
>>> dill.loads(dill.dumps(s))
<socket._socketobject object at 0x10dae18a0>
>>> dill.loads(dill.dumps(w))
<Queue.Queue instance at 0x10db49f38>
>>>
>>> from processing import Queue
>>> q = Queue()
>>> dill.loads(dill.dumps(q))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.2.dev-py2.7.egg/dill/dill.py", line 180, in dumps
    dump(obj, file, protocol, byref, file_mode, safeio)
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.2.dev-py2.7.egg/dill/dill.py", line 173, in dump
    pik.dump(obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/Users/mmckerns/lib/python2.7/site-packages/processing/queue.py", line 62, in __getstate__
    assertSpawning(self)
  File "/Users/mmckerns/lib/python2.7/site-packages/processing/forking.py", line 24, in assertSpawning
    'processes through inheritance' % type(self).__name__)
RuntimeError: Queue objects should only be shared between processes through inheritance

, __getstate__ multiprocessing Queue . , , .

multiprocessing Queue pickle .

+2

More articles:


All Articles