How can I provide good coverage testing of my big Python proejct

Question

I have a very large python project with a very large test suite. Recently, we decided to quantify the quality of our test coverage.

I am looking for a tool to automate the generation of a test report. Ideally, I would like to have attractive, easy-to-read reports, but I would agree to less attractive reports if I could make it work fast.

I tried Nose, which is not good enough: it is not compatible with the distribute / setuptools spatial name package function. Unfortunately, covering the nose will never work for us, since we use this feature. This is a real shame because Nose seems to work very well in Hudson (mostly)

Alternatively, I heard that there is a way to do Python coverage analysis in Eclipse, but I did not completely block the perfect technique.

Any suggestions are welcome!

FYI we use Python 2.4.4 on Windows XP 32bit

Answer 1

Have you tried using coverage.py ? It underlies the "nose covering", but can be perfectly done outside the nose if you need to.

If you run your tests with (hypothetically) python run_my_tests.py , then you can measure coverage with coverage run run_my_tests.py , and then get HTML reports with coverage html .

From your description, I'm not sure what problems you had with your nose, especially whether it was a nose problem or a coating problem. Provide a few details, and I'm sure we can work them out.

Answer 2

Ned already mentioned his excellent coverage.py module.

If the problem you are facing has a specific nose, you might consider using another test runner. I used py.test along with pytest_coverage , which allows you to create coverage statistics. It also has a pytest_nose plugin that helps you do the migration.

However, I do not understand what the problem is that you are facing. Can you talk a little about the function of the distribute / setuptools package for names? I am curious to know what the problem is.