Loading a pickled object into another file - Attribute Error

Question

I have some problems loading the pickled file in a module, which is different from the module where I pickled the file. I am aware of the following stream: Unable to upload files using pickle and multi-component modules . I tried the proposed solution for importing a class into a module, where I print my file, but it continues to give me the same error: AttributeError: Can't get attribute 'Document' on <module '__main__' from ''>

The basic structure of what I'm trying to do is:

A utility file that parses and unpacks objects, utils.py:

 import pickle def save_document(doc): from class_def import Document write_file = open(file_path, 'wb') pickle.dump(doc, write_file) def load_document(file_path): from class_def import Document doc_file = open(file_path, 'rb') return pickle.load(doc_file) 

The file in which the Document object is set and the save utility method, class_def.py is called:

 import utils class Document(object): data = "" if __name__ == '__main__': doc = Document() utils.save_document(doc) 

The file where the load loading method is called, process.py:

 import utils if __name__ == '__main__': utils.load_document(file_path) 

Running process.py gives the mentioned AttributeError attribute. If I import the class_def.py file into process.py and run its main method as indicated in the source stream, it works, but I want to be able to run these two modules separately, since the class_def file is a preprocessing step that takes quite when something. How can i solve this?

Answer 1

in your class_def.py file you have this code:

 if __name__ == '__main__': doc = Document() utils.save_document(doc) 

This means that the doc will be a __main__.Document object, so when it is pickled, it expects to be able to get the Document class from the main module, to fix this, you need to use the Document definition from the module called class_def , which means you would add import here:

 if __name__ == '__main__': from class_def import Document # ^ so that it is using the Document class defined under the class_def module doc = Document() utils.save_document(doc) 

thus, it will have to run the class_def.py file twice, once as __main__ and once as class_def , but this means that the data will be pickled as a class_def.Document object, so the class will be retrieved from the right place. Otherwise, if you have a way to create one document object from another, you can do something like this in utils.py :

 def save_document(doc): if doc.__class__.__module__ == "__main__": from class_def import Document #get the class from the reference-able module doc = Document(doc) #convert it to the class we are able to use write_file = open(file_path, 'wb') pickle.dump(doc, write_file) 

Although usually I prefer the first method.

Answer 2

I had a similar problem and I just realized the differences between our implementations.

Your file structure:

• util.py
  • determine brine functions
• class_def.py
  • import util
  • define class
  • make an instance
  • save pickle call
• process.py
  • import util
  • pickle for cargo

My mistake (using your file names) was the first:

• util_and_class.py
  • define class
  • determine brine functions
  • make an instance
  • save pickle call
• process.py
  • import util_and_class
  • pickle loading call <<ERROR

What solved the pickle import problem:

• util_and_class.py
  • define class
  • determine brine functions
• pickle_init.py
  • import util_and_class
  • make an instance
  • save pickle call
• process.py
  • pickle loading call

This had a welcome side effect that I didn't have to import the util_and_class file as it was baked into the pickle file. Calling the instance and saving the brine in a separate file solved the problem __name__ "loading the pickled file into a module that is different from the module in which I pickled the file."