Coverage of a frozen executable

Question

Is there a way to start coverage for an executable created using pyinstaller? I tried just running it, like it was a python script, and it didn’t like the executable as input (I really didn’t expect it to work), and I suspect there is no answer, there is no easy way to launch coverage against the built executable. ... (this is on windows.exe)

The coverage package I use is the regular coverage package that you get with the "easy_install coverage" from nedbatchelder.com ( http://nedbatchelder.com/code/coverage/ )

Answer 1

This is not a fully formulated answer, but what I have found so far.

From my understanding of how pyinstaller works, it is that the binary is created from a small C program that has a python interpreter and boot files that load the script. PyInstaller, built by EXE, includes an archive after the end of the actual binary file that contains resources for python code. This is explained here http://www.pyinstaller.org/export/develop/project/doc/Manual.html#pyinstaller-archives .

There is iu.py from Pyinstaller / loader / iu.py docs . You should be able to create an import hook to import from a binary file. A breakdown for the pyinstaller disassembler is found https://bitbucket.org/Trundle/exetractor/src/00df9ce00e1a/exetractor/pyinstaller.py , which seems to be able to extract the necessary parts.

Another part of this is that all resources in the binary archive will be compiled with python code. Most likely, cover.py will give you useless output in the same way as when you hit any other compiled module under normal conditions.

Answer 2

Select cover_pylib=True

I know this is long after you asked the question, but I just get around to answering. :)

Using the current source package for coverage.py , I can successfully collect coverage data from the PyInstaller EXE file I created.

In the main source for my application, I conditionally say that lighting will begin to collect coverage as follows:

 if os.environ.has_key('COVERAGE') and len(os.environ['COVERAGE']) > 0: usingCoverage = True import coverage import time cov = coverage.coverage(data_file='.coverage.' + version.GetFullString(), data_suffix=time.strftime(".%Y_%m_%d_%H_%M.%S", time.localtime()), cover_pylib=True) cov.start() 

It starts to collect the collection ONLY when I wish. Using data_suffix allows me to more easily use cov.combine() to merge the file system later. version.GetFullString() is just the version number of my applications.

cover_pylib is set to True here because all the standard __file__ attributes of the __file__ library look like ...\_MEIXXXXX\random.pyc and thus are indistinguishable (along the way) from other code that does not exist inside the package.

When the application is ready to complete, I have this small snippet:

 if usingCoverage: cov.stop() cov.save() 

Once my application is launched, service.py will still not automatically generate its own HTML report for it. Coverage data must be cleared so that references to ...\_MEIXXXX\... are converted to absolute paths to the real source code.

I do this by running this piece of code:

 import sys import os.path from coverage.data import CoverageData from coverage import coverage from glob import glob def cleanupLines(data): """ The coverage data collected via PyInstaller coverage needs the data fixed up so that coverage.py report generation code can analyze the source code. PyInstaller __file__ attributes on code objecters are all in subdirectories of the _MEIXXXX temporary subdirectory. We need to replace the _MEIXXXX temp directory prefix with the correct prefix for each source file. """ prefix = None for file, lines in data.lines.iteritems(): origFile = file if prefix is None: index = file.find('_MEI') if index >= 0: pathSepIndex = file.find('\\', index) if pathSepIndex >= 0: prefix = file[:pathSepIndex + 1] if prefix is not None and file.find(prefix) >= 0: file = file.replace(prefix, "", 1) for path in sys.path: if os.path.exists(path) and os.path.isdir(path): fileName = os.path.join(path, file) if os.path.exists(fileName) and os.path.isfile(fileName): file = fileName if origFile != file: del data.lines[origFile] data.lines[file] = lines for file in glob('.coverage.' + version.GetFullString() + '*'): print "Cleaning up: ", file data = CoverageData(file) data.read() cleanupLines(data) data.write() 

The for loop here is intended solely to clear all coverage files that will be merged.

NOTE. The only coverage data that this code does not execute by default is PyInstaller related files that do not include _MEIXXX data in their __file__ attributes.

Now you can successfully generate HTML or XML (or something else) coverage.py report in the usual way.

In my case, it looks like this:

 cov = coverage(data_file='.coverage.' + version.GetFullString(), data_suffix='.combined') cov.load() cov.combine() cov.save() cov.load() cov.html_report(ignore_errors=True,omit=[r'c:\python27\*', r'..\3rdParty\PythonPackages\*']) 

Using data_file in the constructor is to ensure that load / comb will correctly recognize all of my cleaned coverage files.

The html_report call tells coverage.py ignore the standard python libraries (and the Python libraries checked in my version control tree) and focus only on my application code.

Hope this helps.