Geek Answers Handbook

Organization of the project with Cython and C ++

I want to provide my project in C ++ using the Python interface. Technically, I decided to use Cython to package C ++ code. Over time, the entire project should become a Python extension module, but at first it is very experimental. Gradually, C ++ classes should be exposed to Python.

My question is how best to organize files and create configurations so that the Cython-generated and human-written C ++ code does not mix, and the Python extension module is cleanly built separately from other goals.

I present a directory structure similar to this for the source files, and some build directory for Cython.

Project/ src/ *.h *.cpp cython/ Project.pyx setup.py
+8
c ++ python build cython
source share
1 answer

I basically have 3 folders:

  • CPROJECT , C ++ library: creating a libcproject.so shared object
  • CYPROJECT . Cited Python extension: creating cyproject.so with Cython
  • DEPENDENCIES , dependencies: where I copy external requirements for both projects

In 1. I create a C ++ extension (compiled with gcc compilation options - -shared , -fPIC ) that will be displayed in python and that CYPROJECT relies on Python function expansion. As a post-processing command, the resulting .so copied to DEPENDENCIES/libcproject/ (as well as include files). Thus, the library, of course, can be used independently and in a clean C ++ project.

In 2. I use 3 subfolders:

  • adapters : basically contains additional C ++ classes (often classes derived from those provided by libcproject.so ). Typically, these are classes that are complemented by functionality specific to Cython requirements (for example, saving the version of PyObject * C of the target version of Python - inherited from object - this class and controlling link counting through Py_XINCREF and Py_DECREF , ...).
  • pyext : where all Cython files written by .pyx .
  • setup : contains setup.sh script (for setting dependency paths and calling python setup.py build_ext --inplace to create the final cyproject.so (for adding to PYTHONPATH ) and cyproject.pyx .

So what's in the setup subfolder?

Here is the sample code for setup.sh :

 export PYTHONPATH=$PYTHONPATH:../../../DEPENDENCIES/Cython-0.18 export PATH=$PATH:../../../DEPENDENCIES/libcproject:../../../DEPENDENCIES/Cython-0.18/bin # Note the `../../../DEPENDENCIES/libcproject`... CC="gcc" \ CXX="g++" \ python setup.py build_ext --inplace

And here is the setup.py example (mainly to demonstrate how the extra adapters are compiled):

 import sys import os import shutil from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext # Cleaning for root, dirs, files in os.walk(".", topdown=False): for name in files: if (name.startswith("cyproject") and not(name.endswith(".pyx"))): os.remove(os.path.join(root, name)) for name in dirs: if (name == "build"): shutil.rmtree(name) # Building setup( cmdclass = {'build_ext': build_ext}, ext_modules = [ Extension("cyproject", sources=["cyproject.pyx", \ "adapter/ALabSimulatorBase.cpp", \ "adapter/ALabSimulatorTime.cpp", \ "adapter/ALabNetBinding.cpp", \ "adapter/AValueArg.cpp", \ "adapter/ALabSiteSetsManager.cpp", \ "adapter/ALabSite.cpp", \ ], libraries=["cproject"], language="c++", extra_compile_args=["-I../inc", "-I../../../DEPENDENCIES/python2.7/inc", "-I../../../DEPENDENCIES/gsl-1.8/include"], extra_link_args=["-L../lib"] extra_compile_args=["-fopenmp", "-O3"], extra_link_args=[] ) ] )

And finally, the main .pyx , which binds all the hand-written .pyx parts of cython together [ cyproject.pyx ]:

 include "pyext/Utils.pyx" include "pyext/TCLAP.pyx" include "pyext/LabSimulatorBase.pyx" include "pyext/LabBinding.pyx" include "pyext/LabSimulatorTime.pyx" ...

Note. All files created by Cython remain in this setup folder, well separated from hand-written files ( adapters and pyext ), as expected.

B 3. Using the shared folder DEPENDENCIES allows DEPENDENCIES to keep things well separate (in case I moved CYPROJECT - and its dependencies - to another environment).

All this gives you an overview (in my opinion) how to organize such a project.

+5
source share

More articles:


All Articles