Configuring add-ons using distutils / setuptools

Question

I have a Python project with several extension modules written in C that communicate with a third-party library. However, depending on the user environment and parameters, some modules should not be created, and some compiler flags should be enabled / disabled. The problem is that I need to create a list of extension modules before I call setup (), and ideally I would like to use the distutils.Command subclass to handle custom parameters. Now I have several options:

• Require the python setup.py configure command to be executed before creating the modules, save the information in the pickle file and use it to create a list of extensions the next time the script is run. This is how my project works, which seems pretty dumb.

• Manually clear the parameters from sys.argv and use them to create a list. This is not a long-term solution, because in the end I want to run some scripts to check the settings before creating them.

• A subclass of build_ext from distutils, do my configuration at the beginning of the run () method (possibly also using parameters sent via (2)), and directly modify self.distribution.ext_modules before creating it. I'm afraid this might confuse setuptools, however, since it might suggest that the list of extension modules gets fixed when calling setup (). It also means that when setup () is called with a command other than build_ext, the list of extension modules is empty.

Is there a preferred way to do this?

Answer 1

Is there a preferred way to do this?

In my experience with other people's modules, I can say that there is certainly no consensus on the right path for this.

I tried and rejected the subclassifying bits of distutils - I found it fragile and difficult to work on different versions of Python and different systems.

In our code, after checking the types of things you are considering, I decided to install and configure directly in setup.py before the main setup() call. This is admittedly a little ugly, but it means that someone who is trying to compile your stuff has one place to figure out, for example. why the inclusion path is wrong. (And they, of course, don't need to be experts on internals of distutils).

Answer 2

My own experience with modifying distutils was weak and shaky, so all I can offer is pointers. Look at numpy. This has a whole submodule (numpy.distutils) with distutils working (or working) methods. Otherwise, specify the distutils mailing list.

Answer 3

I would subclass distutils.core.Distribution and pass it using distutils.core.setup(distclass=CustomDistribution) - this gives you access to command line options just like a regular installation, and you can do things like list customization extensions in CustomDistribution.__init__ . But I agree with dalke, the distutils path is filled with pain ...

Answer 4

The config command is for a subclass and is used by projects with requirements like yours.