Insert pickle (or arbitrary) data in python script

Question

In Perl, an interpreter type stops when it encounters a string with

__END__ 

. This is often used to embed arbitrary data at the end of a perl script. Thus, a perl script can retrieve and store data that it stores "by itself", which allows you to get pretty nice features.

In my case, I have a pickled object that I want to store somewhere. Although I can use file.pickle just fine, I was looking for a more compact approach (easier to distribute the script).

Is there a mechanism that allows you to somehow embed arbitrary data inside a python script?

Answer 1

If the data is not particularly large (many Ks), I would simply .encode ('base64') and include a string with .decode ('base64') in the triple quote to return the binary data, and call pickle.loads () around it .

Answer 2

With pickle, you can also work directly with strings.

 s = pickle.dumps(obj) pickle.loads(s) 

If you combine this with "" (triple quotation marks), you can easily store any pickled data in your file.

Answer 3

In Python, you can use "" (triple quotation marks) to embed long lines of text data in your program.

In your case, however, do not waste time on this.

If you have an object that you pickled, you would have been much, much happier dumping that object as a Python source and just turning on the source.

The repr function, which applies to most objects, returns the original version of the Python object. If you implement __repr__ for all your custom classes, you can trivially reset your structure as a Python source.

If, on the other hand, your pickled structure started as Python code, just leave it as Python code.

Answer 4

I made this code. You run something like python comp.py foofile.tar.gz, and it creates decomp.py, with the contents of foofile.tar.gz embedded in it. I do not think it is really portable with windows because of Popen.

 import base64 import sys import subprocess inf = open(sys.argv[1],"r+b").read() outs = base64.b64encode(inf) decomppy = '''#!/usr/bin/python import base64 def decomp(data): fname = "%s" outf = open(fname,"w+b") outf.write(base64.b64decode(data)) outf.close() # You can put the rest of your code here. #Like this, to unzip an archive #import subprocess #subprocess.Popen("tar xzf " + fname, shell=True) #subprocess.Popen("rm " + fname, shell=True) ''' %(sys.argv[1]) taildata = '''uudata = """%s""" decomp(uudata) ''' %(outs) outpy = open("decomp.py","w+b") outpy.write(decomppy) outpy.write(taildata) outpy.close() subprocess.Popen("chmod +x decomp.py",shell=True)