Geek Answers Handbook

How to handle C extensions for python applications using pip?

For python applications that are installed using pip , how can you automatically handle your C extension requirements?

For example, the mysqlclient module requires the MySQL development libraries installed on the system. When you initially install an application that requires this module, it will fail if the MySQL development libraries are not on the system. So the question is, how can I solve this?

  • Is there any way to solve this with setup.py that I don't know about?
  • Should I not use a pure python module implementation?

Note; I am not looking for answers like " just use py2exe ".

+7
python deployment
source share
2 answers

No It is not possible to include a completely separate C library in the build process if you are not writing an extension . Even so, you will need to specify all .c files in ext_modules so that they can be compiled as part of your build process, which, I know, is not what you need.

The only thing you can do is simply stop the build process and give the user a reasonable error if mysql-devel (or libmysqlclient-dev ) is not already installed.

One way to find out if mysql-dev is installed is to write a simple C function that imports mysql.h and check if it compiled successfully.

Note: mysql.h and my_global.h are part of the libmysqlclient-dev package.

Test / test_mysqlclient.c

 // Taken from: http://zetcode.com/db/mysqlc #include <my_global.h> #include <mysql.h> int main(int argc, char **argv) { printf("MySQL client version: %s\n", mysql_get_client_info()); exit(0); }

Secondly, let's update our setup.py file so that it is included in the build process.

setup.py

 #!/usr/bin/env python import os import subprocess from setuptools import setup, Extension def mysql_test_extension(): process = subprocess.Popen(['which', 'mysql_config'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) result, error = process.communicate() if process.returncode > 0: raise RuntimeError(error) config_command = result.strip() cflags = subprocess.check_output([config_command, '--cflags'], close_fds=True).strip() include_dirs = [] extra_compile_args = [] for arg in cflags.split(' '): if not arg.strip(): continue elif arg.startswith('-I'): include_dirs.append(arg[2:]) elif arg.startswith('-'): extra_compile_args.append(arg) else: extra_compile_args[-1] = extra_compile_args[-1] + ' ' + arg libs = subprocess.check_output([config_command, '--libs'], close_fds=True).strip() libraries = [] linkers = [] for arg in libs.split(' '): if not arg.strip(): continue elif arg.startswith('-L'): libraries.append(arg[2:]) elif arg.startswith('-'): linkers.append(arg) else: linkers[-1] = extra_compile_args[-1] + ' ' + arg return Extension('test_mysqlclient', ['test/test_mysqlclient.c'], include_dirs=include_dirs, library_dirs=libraries, extra_link_args=linkers, extra_compile_args=extra_compile_args) setup(name='python-project', version='1.0', description='Python Project', classifiers=[ 'Development Status :: 5 - Production/Stable', 'Environment :: Console', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python :: 2.7', 'Natural Language :: English', ], keywords='mysql python project', author='Ozgur Vatansever', url='http://github.com/ozgur/python-project/', license='MIT', packages=['some_project'], ext_modules = [mysql_test_extension()] )

You can start building the package with the test_mysqlclient file:

 $ python setup.py build

If mysql-devel is not installed on your system, you will get a build error like this:

 test/test_mysqlclient.c:3:10: fatal error: 'my_global.h' file not found #include <my_global.h> ^ 1 error generated.
+7
source share

So the question is, how can I solve this?

You have not solved this problem. There is no way in setup.py to describe external dependencies outside the python ecosystem. Just specify it in README.

-2
source share

More articles:


All Articles