Python C-extension - returns Py_BuildValue () memory leak problem

Question

I have a huge memory leak problem related to the C extension I am developing. In C, I have an array of doubles called A , and an int variable called AnotherIntVariable , which I want to pass to Python. Well, in my C extension module, I do the following:

 int i; PyObject *lst = PyList_New(len_A); PyObject *num; if(!lst) return NULL; for(i=0;i<len_A;i++){ num=PyFloat_FromDouble(A[i]); if(!num){ Py_DECREF(lst); return NuLL; } PyList_SET_ITEM(lst,i,num); } free(A); return Py_BuildValue("Oi",lst,AnotherIntVariable) 

So, in Python, I get this list and int as follows:

 Pyt_A,Pyt_int=MyCModule.MyCFunction(...) 

Where Pyt_A and Pyt_int are the list and integer obtained from my C-extension " MyCModule ", from the function " MyCFunction ", which I described earlier.

The problem is that in Python I use this Pyt_A array (so I use Py_BuildValue instead of a simple return to make INCREF to save this variable momentarily from the garbage collector), but then I need to dereference it somehow to free allocated memory. The problem is that I use the MyCFunction function several times, and this causes a memory leak, because I do not know how to dereference the array I get in python to get rid of it.

I tried just returning the array by doing return lst in the C part of the code instead of Py_BuildValue("Oi",lst,AnotherIntVariable) , but this only leads to a segmentation error when I try to use it in python (possibly because the garbage collector did his work) ...

... what am I missing here? Can anybody help me?

Answer 1

If you look at the documentation for Py_BuildValue ( http://docs.python.org/3/c-api/arg.html#c.Py_BuildValue ), you will see that in O typecode this says that the reference count of the passed object is incremented by unit (Note: the earlier section of this page describes the O typecode for PyArg_ParseTuple , which does not increment the reference count, but also does not matter here).

So, after calling Py_BuildValue , there will be 2 for your list, but you want it to be 1 .

Instead of directly returning the result of Py_BuildValue , save it to the PyObject pointer, decrease the link count of lst , and then return the result.

In any case, you should check the result of calling Py_BuildValue , since you also need to free num in case Py_BuildValue (i.e. returns NULL ).

Answer 2

Thank you for clearing Ignacio, now it makes such a sense! Finally, the solution was to instead of directly returning Py_BuildValue, execute:

 free(A); PyObject *MyResult = Py_BuildValue("Oi",lst,AnotherIntVariable); Py_DECREF(lst); return MyResult 

It worked like a charm!