Ipyparallel error when using decorator

I am trying to complete a job using the ipyparallel system with lru_cache and lru_cache confusing the problems.

From terminal:

 ipcluster start -n 2 

In ipython laptop:

 from ipyparallel import Client clients = Client() def world(): return hello() + " World" def hello(): return "Hello" world() 'Hello World' 

Running using ipyparallel requires this:

 clients[:].push(dict(hello=hello)) 

Without the previous line, a failure occurs that is not unexpected, but works fine if it is running:

 clients[:].apply_sync(world) ['Hello World', 'Hello World'] 

Everything works as expected, however with lru_cache parallel step generates errors

 from ipyparallel import Client from functools import lru_cache clients = Client() def world(): return hello() + " World" @lru_cache(maxsize=2048) def hello(): return "Hello" clients[:].push(dict(hello=hello)) clients[:].apply_sync(world) 

This fails with the following error:

  [0:apply]: ---------------------------------------------------------------------------NameError Traceback (most recent call last)<string> in <module>() <ipython-input-17-9ac5ef032485> in world() NameError: name 'hello' is not defined [1:apply]: ---------------------------------------------------------------------------NameError Traceback (most recent call last)<string> in <module>() <ipython-input-17-9ac5ef032485> in world() NameError: name 'hello' is not defined 

I understand that this could be a namespace problem where lru_cache uses closure, however I hope there is a workaround so that I can use it. Or, if someone tells me that this is simply not possible, that would also be useful, thanks.

I use:
ipykernel (4.1.1)
ipyparallel (4.1.0)
ipython (4.0.1)
Python (3.5.1)

+6
source share

All Articles