Python C Extensions - Why Callable C Functions Take Arguments and Return PyObject *

Question

I'm just starting to play with Python C extensions, and I'm curious why the C function called from Python should take 2 arguments of PyObject * and return PyObject *. I wrote the following Hello World extension:

#include <Python.h> static PyObject * hello_world(PyObject *self, PyObject *noargs) { printf("Hello World\n"); return Py_BuildValue(""); } // Module functions table. static PyMethodDef module_functions[] = { { "hello_world", hello_world, METH_NOARGS, "hello world method" }, { NULL } }; // This function is called to initialize the module. PyMODINIT_FUNC inittesty2(void) { Py_InitModule("testy2", module_functions); } 

Why can't I (especially with METH_NOARGS) use the following hello_world method:

 static void hello_world() { printf("Hello World\n"); } 

?

Answer 1

There are a few things that can be said about the various pointers of PyObject .

• The one required for the return type is used for the processing engine . . In particular, if your function returns a null pointer, the Python interpreter will throw an exception. (You should only do this after calling one of the PyErr_.. functions to set a specific exception.)

  This also means that whenever you do not want to throw an exception, you should return a pointer to some real PyObject . If there is nothing, in particular, your function should return, simply return Py_None (it is best to use the Py_RETURN_NONE macro to get the correct number of links) or "true" (using Py_RETURN_TRUE ).

• First argument , PyObject *self points to the object from which the function is called, or to the instance of the module to which it belongs. Note that each function you assign is a class method or a module method. There are no completely independent functions.

• The second argument, PyObject *args points to the function argument (which may be a tuple or a list of several arguments). You are right in pointing out that a function that takes no arguments does not need it - and as far as I can tell, you are right. You must not define this; you can simply define the function as

   static PyObject *PyMyClass_MyFunc(PyObject *self) { /* ..do something.. */ Py_RETURN_TRUE; } 

  You still have to pass this PyCFunction when you put it in PyMethodDef for the data type that you define, but I think casting is safe while you use METH_NOARGS . But look at the comments below for possible risks.

• Finally, the function may actually have a third argument :

   static PyObject *PyMyClass_Func(PyObject *self, PyObject *args, PyObject *kwds) { /*...*/ } 

  The third argument is used for named optional arguments. In this case, you should also point to the PyCFunction , but it is also safe if you set the correct flag ( METH_KEYWORDS ) for your function in the method table.

Answer 2

The first argument for module level functions is the module object. When defining classes in C (the same PyMethodDef structure is used for the same methods), the first argument is an instance (similar to self in Python).

When using METH_NOARGS , Python will pass NULL as the second argument. They could pass it functions with one argument, but I think they did not think that it was necessary.

The return value is easy to explain. Each Python function has a return value. If you are not explicitly using return in Python, the function will return None .

Of course, in C you should be explicit about the return value, so if you are not using it, you need to return None yourself. Python provides a macro for this:

 Py_RETURN_NONE; 

Alternatively, you can access the global instance of None yourself:

 Py_INCREF(Py_None); return Py_None; 

but a macro is easier to use.

You might think that returning NULL should be equivalent to None , but NULL used to indicate that the function raised an exception.

Answer 3

Since Python functions, even trivial ones that just print to stdout, are more than just wrappers around C. functions

In the simplest case, think of introspection tools in Python. The Python function is a fully functional object that you can request:

 >>> def hello(): ... print 'hello' ... >>> dir(hello) ['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name'] 

You could, of course, imagine an extension tool that simply wrapped C functions. Check out SWIG , which allows you to expand entries for a large number of scripting languages, including Python. Since it will be used for the lowest common denominator, it will allow you to wrap a function like your hello_world , but of course you will lose a lot of energy too.