Does it make sense to install my Python unit tests in site packages?

Question

I am developing my first Python distribution. My learning curve for the Python Package seems to align a bit, but I'm still struggling with a few open questions. One of them is whether I should trigger my unit tests installed next to my code.

I understand it is important to include tests in the source distribution . I am wondering if I should configure them for installation?

I have seen at least one popular package that seems to be doing this on purpose ( PyHamcrest ), and at least one other that seems to be doing it by accident ( behave ).

So my (multi-part) question is this:

• Does it make sense to install my unit package tests along with my package code?

• If so, what is the precedent? Who will use them and for what? That is, who will use them who will not be happy to load the distribution source and run python setup.py test instead?

• And how will they use the installed unit tests? like import test; test.run() import test; test.run() or something like this?

Answer 1

After studying this problem, and until someone more experienced can not look in the opposite direction, I understand that the simple answer is: "No, unit tests should not be installed, included only in the source distribution."

In several cases, I found that the tests were installed, all turned out to be random , and easier than you might think, to make a mistake without noticing it.

Here's how it goes:

• The packages=find_packages() parameter is used in the setup.py file, so packages can be found without explicitly specifying.
• The test folder turns into a package (by adding __init__.py ), so tests can refer to the modules they test using relative naming (for example, from .. import pkg.mod ).
• setuptools installs test as a separate package along with others in the project. Please note, this means that you can run the import test in the python interpreter, and it works, almost certainly not what you intended, especially since many other people use this name for their test directory :)

The fix is ​​to use the parameter: packages=find_packages(exclude=['test']) to prevent the installation of the test directory.

Answer 2

In my opinion, the correct answer is NO, but you will find many distributions that install tests. Tests should not be installed, but they should be included in the source distribution. In my opinion, in an ideal world, testing installed packages should be the task of the package manager (pip), and the site-packages directory should not be contaminated with testing sources.

I recently researched this topic and gathered information from various sources and found several different ways to structure the directory / package hierarchy of a distribution that contains both library sources and tests. Most of these structures seem to be outdated, and they were invented as attempts to get around incomplete sets of old distribution systems at the time. Unfortunately, many online sources (old blog pages / documentation) still advertise outdated methods, so it’s very easy to find an outdated usage guide / tutorial with online search.

Suppose you have a library called "my_lib" and want to structure your distribution sources. I will show you two popular and seemingly outdated ways to structure your distribution, and the third method, which I consider the most universal. The third method may also be deprecated, but the very best I know is the publication time of this answer .; -)

Method # 1

Distributions that (intentionally or unintentionally) install tests typically use this method.

hierarchies

 +- my_lib | +- __init__.py | +- source1.py | +- source2.py | +- tests | +- __init__.py | +- test_1.py | +- test_2.py +- setup.py 

Method # 2

Tests are not installed, but they must be included in the source distribution through the MANIFEST.in file.

hierarchies

 +- my_lib | +- __init__.py | +- source1.py | +- source2.py +- tests | +- __init__.py | +- test_1.py | +- test_2.py +- setup.py 

Method number 3 (I prefer this one).

This is very similar to method # 2 with a slight twist ( src dir).

hierarchies

 +- src | +- my_lib | +- __init__.py | +- source1.py | +- source2.py +- tests | +- __init__.py | +- test_1.py | +- test_2.py +- setup.py 

setup () call in setup.py

 from setuptools import setup, find_packages setup( ... packages=find_packages('src'), package_dir={'': 'src'}, ... ) 

MANIFEST.in

 recursive-include tests *.py 

Tests will not be installed, but they will be included in the source distribution through our MANIFEST.in .

For method # 3, you have the src directory, which usually contains only one package, which is the root of your library. Putting the my_lib package in the src directory (a directory, not a package, so you don't need src/__init__.py ) has the following advantages:

• When you execute setup.py , the directory containing setup.py is implicitly added to the python path. This means that in setup.py you can accidentally and incorrectly import material from your library if the package is in the same directory as setup.py . By placing the my_lib package in src , we can avoid this problem.

• You can easily use distributed test sources to test both the sources of the distributed library and the installed library:

  • When you run tests using setup.py test , the package_dir={'': 'src'} your setup() call ensures that your tests see your my_lib library package, which you store in src/my_lib .
  • You can also run tests without setup.py . In the simplest case, you can do this with the python -m unittest . In this case, dir src will not be part of the python path, so you can use this method to check the installed version of the library instead of sources in src .

Answer 3

However, I am not an expert, I would like to share my opinion.

I always put tests next to the code if I expect something to fail depending on external reasons. Be it bit-ordering, weird time zones, character encoding, 24-bit integers, or anything else fancy that you may encounter and pass the test.

Who would not be happy to download the source code and run the tests? Perhaps some debian users who package packages from sources (I know you are talking about python, but let me be a little generic), and your library may sometimes fail due to some strange things on the system.

If your tests provide only internal sanity, I would skip them, because using sources they are not worth much, since you will never change the insides of the library.

Personally, I heard that this is not so, because it was transferred to some IBM machine that had a different bit order. I don’t remember whether it depended on a bit operation or something pre-computed and cached statically. But sometimes it’s wise to check if you are uploading what you think you saved.

EDIT: Maybe it's better to rephrase it. I would install tests if you feel that there might be reservations about portability. I find it always useful to check when you are deploying stuff to another system.