Can't pickle the lumberjacks?

Question

Work on a project that requires me to sort a container object at any time, since we expect it to fail on external conditions quite often and be able to consciously choose where we left off.

I pretty often use the python protocol library, and all my classes start with a registrar setup:

class foo: def __init__(self): self.logger = logging.getLogger("package.foo") 

Because I am building a container class, it has several layers of classes, each with its own log instance.

Now, for some reason, these recorders are breaking Pickle. I get the following error, which disappears if I remove self.logger from all classes:

 Can't pickle 'lock' object: <thread.lock object at ... > 

So my question is, is there a way to remove lock objects from all registrars without having to recurse through my entire object tree, deleting the registrars that I have to recreate on unickle.

Answer 1

Found a very similar question here, with the answer that worked for me:

How to stop attributes from pickling in Python

edit: used this answer: How to stop attributes from pickling in Python

Answer 2

You can create a class that wraps the registrar and implements __getstate__ and __setstate__ .

This is pasted from http://docs.python.org/library/pickle.html . fh handled in a way that may be similar to what you need.

 #!/usr/local/bin/python class TextReader: """Print and number lines in a text file.""" def __init__(self, file): self.file = file self.fh = open(file) self.lineno = 0 def readline(self): self.lineno = self.lineno + 1 line = self.fh.readline() if not line: return None if line.endswith("\n"): line = line[:-1] return "%d: %s" % (self.lineno, line) def __getstate__(self): odict = self.__dict__.copy() # copy the dict since we change it del odict['fh'] # remove filehandle entry return odict def __setstate__(self, dict): fh = open(dict['file']) # reopen file count = dict['lineno'] # read from file... while count: # until line count is restored fh.readline() count = count - 1 self.__dict__.update(dict) # update attributes self.fh = fh # save the file object 

Answer 3

You can also create a class that implements a property that returns the required registrar. Each class that inherits from this "LoggerMixin" can now use the logger in the same way as you used to.

 class LoggerMixin(): @property def logger(self): component = "{}.{}".format(type(self).__module__, type(self).__name__) return logging.getLogger(component) class Foo(LoggerMixin): def __init__(self): self.logger.info("initialize class") def bar(self): self.logger.info("execute bar") 

Answer 4

Here you could use a dill that can sort loggers and locks.

 >>> class foo: ... def __init__(self): ... self.logger = logging.getLogger("package.foo") ... >>> import dill >>> import logging >>> >>> f = foo() >>> _f = dill.dumps(f) >>> f_ = dill.loads(_f) >>> f_.logger <logging.Logger object at 0x110b1d250>