Saving the state of the program so that it can be resumed

Question

I sometimes have Python programs that take a lot of time, and I want to save the state and resume it later. Does anyone have a smart way to save state either every x seconds, or when the program exits?

Answer 1

Put all your status data in one place and use pickle .

The pickle module implements a fundamental but powerful algorithm for serializing and de-serializing the structure of a Python object. Etching is the process by which the hierarchy of Python objects is converted to a byte stream, and unpickling is the reverse operation by which the byte stream is converted back to the object hierarchy. However, etching (and transfusion) is alternatively referred to as “serialization”, “sorting”, 1 or “smoothing” to avoid confusion; the conditions used here are “etching” and “spilling”.

Answer 2

If you want to save everything, including the full namespace and the line of code currently being executed to restart at any time, there is no standard library module for this.

As another poster said, the pickle module can save almost everything in the file, and then load it again, but you will need to specifically design your program around the brine module (that is, save your “state” - including variables, etc. . - in the class).

Answer 3

If you're fine with OOP, consider creating a method for each class that outputs a serialized version (using pickle) to a file. Then add the second method to load into the data instance, and if in the pickled file you call the load method instead of the processing one. I use this approach for ML and it really started my workflow.