Indeed, the best answer here is probably to use numpy arrays exclusively, even from your C code. But if this is not possible, then you have the same problem as any code that shares data between C types and Python types.
In general, there are at least five options for sharing data between C and Python:
- Create a Python
list or other object to pass. - Define a new Python type (in your C code) to port and represent the array with the same methods that you define for the sequence object in Python (
__getitem__ , etc.). - Place the pointer to the array in
intptr_t or to the explicit type ctypes or just leave it inactive; then use ctypes on the Python side to access it. - Move the pointer to the array to
const char * and pass it as str (or in Py3, bytes ) and use struct or ctypes on the Python side to access it. - Create an object that conforms to the
buffer protocol, and again use struct or ctypes on the Python side.
In your case, you want to use numpy.array in Python. So, the general cases are:
- Create
numpy.array to jump. - (may not fit)
- Pass a pointer to the as-is array and from Python, use
ctypes to get it into a type that numpy can convert to an array. - Move the pointer to the array to
const char * and pass it as str (or in Py3, bytes ), which is already a type that numpy can convert to an array. - Create an object that conforms to the
buffer protocol, and which, I believe, numpy can directly convert.
For 1, here's how to do it with list , simply because it's a very simple example (and I already wrote it ...):
PyObject *makelist(int array[], size_t size) { PyObject *l = PyList_New(size); for (size_t i = 0; i != size; ++i) { PyList_SET_ITEM(l, i, PyInt_FromLong(array[i])); } return l; }
And here is the equivalent of numpy.array (suppose you can rely on the C array not to be deleted - see Creating arrays in docs for more details on your options here):
PyObject *makearray(int array[], size_t size) { npy_int dim = size; return PyArray_SimpleNewFromData(1, &dim, (void *)array); }
Anyway, no matter how you do it, you will get something that looks like PyObject * from C (and has one refcount), so you can pass it as an argument to a function, and on the Python side it will look like numpy.array , list , bytes or something else suitable.
Now, how do you actually pass arguments to a function? Well, the sample code in Pure Embedding that you referenced in your comment shows how to do this, but doesn't really explain what is going on. There are more explanations in the extension documents than the deployment documents, in particular Calling Python Functions from C. Also, keep in mind that the standard library source code contains many examples of this (although some of them are not as readable as they might have been due to optimization, or simply because they have not been updated to take advantage of the new simplified ones API functions C).
Skip the first example of getting a Python function from Python, because you probably already have this. The second example (and the paragraph on the right) shows an easy way to do this: Creating a tuple of arguments with Py_BuildValue . So, let's say we want to call the function that you saved in myfunc using the mylist returned by this makelist function above. Here is what you do:
if (!PyCallable_Check(myfunc)) { PyErr_SetString(PyExc_TypeError, "function is not callable?!"); return NULL; } PyObject *arglist = Py_BuildValue("(o)", mylist); PyObject *result = PyObject_CallObject(myfunc, arglist); Py_DECREF(arglist); return result;
If you are sure that you have a valid callable, you can skip the call check. (And it's usually best to check when you get myfunc first if necessary, because you can give both earlier and better error feedback in this way.)
If you want to understand what is happening, try it without Py_BuildValue . As the docs say, the second argument [PyObject_CallObject][6] is a tuple, and PyObject_CallObject(callable_object, args) equivalent to apply(callable_object, args) , which is equivalent to callable_object(*args) . So, if you want to call myfunc(mylist) in Python, you should include this accordingly, myfunc(*(mylist,)) so that you can translate it to C. You can build a tuple like this:
PyObject *arglist = PyTuple_Pack(1, mylist);
But usually Py_BuildValue simpler (especially if you haven't packed everything like Python objects yet), and the intent in the code is clearer (just like using PyArg_ParseTuple simpler and more understandable than using an explicit tuple work in a different direction).
So how do you get this myfunc ? Well, if you created a function from the embed code, just hold the pointer. If you want it to go through Python code, this is exactly what the first example does. If you want, for example, to look at it by name from a module or other context, the API for specific types such as PyModule and abstract types such as PyMapping is quite simple, and it’s usually obvious how to convert Python code to equivalent C code, even if the result is basically an ugly pattern.
Putting it all together, let's say I have an array of C integers, and I want to import mymodule and call the function mymodule.myfunc(mylist) , which returns an int. Here's a stripped-down example (not actually tested and not handling the error, but it should show all the parts):
int callModuleFunc(int array[], size_t size) { PyObject *mymodule = PyImport_ImportModule("mymodule"); PyObject *myfunc = PyObject_GetAttrString(mymodule, "myfunc"); PyObject *mylist = PyList_New(size); for (size_t i = 0; i != size; ++i) { PyList_SET_ITEM(l, i, PyInt_FromLong(array[i])); } PyObject *arglist = Py_BuildValue("(o)", mylist); PyObject *result = PyObject_CallObject(myfunc, arglist); int retval = (int)PyInt_AsLong(result); Py_DECREF(result); Py_DECREF(arglist); Py_DECREF(mylist); Py_DECREF(myfunc); Py_DECREF(mymodule); return retval; }
If you use C ++, you probably want to look into some kind of protection / janitor / etc. to handle all of these Py_DECREF calls, especially after you start doing the correct error handling (which usually means that the early return NULL calls passed through a function). If you are using C ++ 11 or Boost, unique_ptr<PyObject, Py_DecRef> may be all you need.
But in fact, the best way to reduce all of these ugly patterns, if you plan to make a lot of C ↔ Python posts, is to look at all the familiar frameworks designed to improve the Python- Cython extension, boost :: python , etc. Even if you are embedding, you are effectively doing the same work as the extension, so they can help in the same way.
In this regard, some of them also have tools that help the embedded part if you search in documents. For example, you can write your main program in Cython using both C code and Python code, and cython --embed . You may want to cross your fingers and / or sacrifice some chickens, but if it works, it is surprisingly simple and productive. Boost is not so empty to start, but as soon as you pack, almost everything is done exactly as you expected, and just works, and it is just as embedding as expanding. And so on.