Can I "clog local objects" if I use a derived class?

Question

The pickle link states that the set of objects that can be pickled is rather limited. In fact, I have a function that returns a dynamically generated class, and I found that I can not saw through instances of this class:

 >>> import pickle >>> def f(): ... class A: pass ... return A ... >>> LocalA = f() >>> la = LocalA() >>> with open('testing.pickle', 'wb') as f: ... pickle.dump(la, f, pickle.HIGHEST_PROTOCOL) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> AttributeError: Can't pickle local object 'f.<locals>.A' 

Such objects are too complicated for pickle . OK. Now, what is magic, if I try to rekindle a similar object, but a derived class, it works!

 >>> class DerivedA(LocalA): pass ... >>> da = DerivedA() >>> with open('testing.pickle', 'wb') as f: ... pickle.dump(da, f, pickle.HIGHEST_PROTOCOL) ... >>> 

What's going on here? If this is so simple, why does pickle not use this workaround to implement the dump method, which allows pickling "local objects"?

Answer 1

I think you have not read the link you are quoting . The link also clearly states that only the following objects are legible:

• defined at the top level of the module (using def, not> lambda)
• built-in functions defined at the top level of the module
• which are defined in the top level of the module

Your example

 >>> def f(): ... class A: pass ... return A 

does not define a class at the top level of the module; it defines a class within the scope of f() . pickle works with global classes, not local classes. This automatically fails the mock test.

DerivedA is a global class, so everything is fine.

As for why top-level classes and functions (global for you) cannot be pickled, this link also answers this question (bold mines):

Please note that functions (built-in and user-defined) are selected using "full qualification" , and not by value. This means that only the function name is zamarino along with the name of the module in which the function is defined. Neither the function code nor any of its function attributes is etched . Thus, the defining module must be imported in the splitting environment, and the module must contain a named object, otherwise an exception will be thrown.

Similarly, classes are pickled by a named link, so the same restrictions apply in a scattering environment.

So you have it. pickle only serializes objects by reference of the name, and not by the raw instructions contained in the object. This is because pickle's job is to serialize the hierarchy of objects and nothing more.

Answer 2

I do not agree, you can pickle both. You just need to use a better serializer like dill . dill (default) pickle classes, storing the class definition instead of pickling by reference, so this will not fail your first case. You can even use dill to get the source code if you want.

 >>> import dill as pickle >>> def f(): ... class A: pass ... return A ... >>> localA = f() >>> la = localA() >>> >>> _la = pickle.dumps(la) >>> la_ = pickle.loads(_la) >>> >>> class DerivedA(localA): pass ... >>> da = DerivedA() >>> _da = pickle.dumps(da) >>> da_ = pickle.loads(_da) >>> >>> print(pickle.source.getsource(la_.__class__)) class A: pass >>> 

Answer 3

DerivedA instances are pickleable because DerivedA is accessible through a global variable corresponding to its fully qualified name, namely how pickle searches for classes when unpacking.

The problem with trying to do something similar with local classes is that there is nothing that determines which class A corresponds to the instance. If you run f twice, you get two classes of A , and there is no way to determine which one should be the class of unpainted instances of A from another program run. If you don't run f at all, you don't get the A classes, and then what do you do about the type of unpainted instances?