Distutils unexpectedly fails with a (working) SWIG extension

Question

I am wrapping a C library in a python module through SWIG, here called myExample. If I compile:

$swig -python myExample.i $gcc -c myExample_wrap.c -I /usr/lib/python2.7 -fPIC -std=c99 $ld -shared myExample_wrap.so -llapacke -o _myExample.so 

I get a full working module (liblapacke is needed for some functions that I used). Now I want to make this module available through pip install.

According to the distutils section ( https://docs.python.org/2.7/distutils/setupscript.html ), I wrote the setup.py file:

 from distutils.core import setup, Extension setup(name='myExample', version='0.1', ext_modules=[Extension('_myExample',['myExample.i'], libraries= ['lapacke'])] ) 

and edited MANIFEST.in in such a way as to save the sources and avoid problems such as similar issues on this website (for example, myExample.h and myExample.c). Then I run:

 $python setup.py sdist 

and got the package installed through "pip install". This was done (no errors, no warnings), but ... it does not work. In this installable module ("_myExample.so" - note the underscore, distutils seems to be required [maybe it hides the answer?]) Some methods are different, some are missing, etc. So I decided one step at a time. Just compiling:

 $python setup.py build_ext 

I already got the same problem: the last module is different from the one that was obtained using the usual compilation described at the beginning.

To summarize: given the SWIG interface, compiling it traditionally or compiling through distutils gives a different result. How is this possible? Is my setup.py wrong? Maybe an alternative way to get a module installed on pip without relying on distutils or setuptools (which creates the same problems)?

Ps: the wrapping code is very long, so I can not, unfortunately, provide a detailed list, but I am fully available when adding more if necessary. For example, a manually compiled interface successfully contains “AdaptiveInterpolation” (works fine), while a manufactured distutil has “AdaptiveInterpolation_set”, “AdaptiveInterpolation_get” or there are many methods starting with “new_” (the code is missing in my original).

Answer 1

In practice, there are two distribution options for such a package: bdist_wheel and sdist .

Take the SWIG example from docs for distutils .

example.h

 int fact(int n); 

example.c

 #include "example.h" int fact(int n) { if (n == 0) { return 1; } else { return n * fact(n - 1); } } 

example.i

 %module example %{ #define SWIG_FILE_WITH_INIT #include "example.h" %} int fact(int n); 

Let him build the SWIG interface, which by default creates example_wrap.c .

 swig -python example.i 

Wheel

Python Wheels is a modern distribution format that simplifies the distribution of ready-made packages (i.e. package users do not need a compiler, development headers, etc.). To use it, we need setuptools and wheel (can be installed using pip or from the OS repositories, sudo apt-get install python-wheel python-setuptools ).

setup.py

 from setuptools import setup, Extension setup( name = 'example', version = '0.1', author = 'SWIG Docs', description = 'Simple swig example from docs', ext_modules = [ Extension('_example', sources = ['example_wrap.c', 'example.c'])], py_modules = ['example'], package_data = {'': ['example.h']} # needed for sdist ) 

You can create a wheel with:

 python setup.py bdist_wheel 

On my machine, it creates example-0.1-cp27-cp27mu-linux_x86_64.whl , which I can install with pip and check as python -c 'import example; print(example.fact(5))' python -c 'import example; print(example.fact(5))' . Pay attention to the file name. It encodes a compatible version of Python, ABI, and platform. Here's a listing of the contents ( unzip -l ... ).

 Archive: example-0.1-cp27-cp27mu-linux_x86_64.whl Length Date Time Name --------- ---------- ----- ---- 2609 2018-02-24 11:06 example.py 70968 2018-02-24 13:31 _example.so 10 2018-02-24 14:58 example-0.1.dist-info/DESCRIPTION.rst 290 2018-02-24 14:58 example-0.1.dist-info/metadata.json 17 2018-02-24 14:58 example-0.1.dist-info/top_level.txt 105 2018-02-24 14:58 example-0.1.dist-info/WHEEL 193 2018-02-24 14:58 example-0.1.dist-info/METADATA 617 2018-02-24 14:58 example-0.1.dist-info/RECORD --------- ------- 74809 8 files 

To create wheels with better compatibility, you can, for example, look at manylinux .

A source

sdist means source distribution, which means your users will need to have a compiler and corresponding development dependencies. Source distribution is built using:

 python setup.py sdist 

The example-0.1.tar.gz contains ( tar -tf ... ):

 example-0.1/ example-0.1/example.c example-0.1/PKG-INFO example-0.1/example.egg-info/ example-0.1/example.egg-info/PKG-INFO example-0.1/example.egg-info/dependency_links.txt example-0.1/example.egg-info/top_level.txt example-0.1/example.egg-info/SOURCES.txt example-0.1/setup.cfg example-0.1/example.py example-0.1/setup.py example-0.1/example_wrap.c example-0.1/example.h