How does __call__ work?

Question

The Python __call__ magic method is called whenever you try to call an object. Cls()() is thus equal to Cls.__call__(Cls()) .

Functions are first class objects in Python, that is, they are just callable objects (using __call__ ). However, __call__ itself is a function, so it also has __call__ , which again has its own __call__ , which again has its own __call__ .

So Cls.__call__(Cls()) is equal to Cls.__call__.__call__(Cls()) and is aligned again to Cls.__call__.__call__.__call__(Cls()) , etc. etc.

How does this endless cycle end? How does __call__ really execute code?

Answer 1

Under the hood, all calls in Python use the same mechanism, and almost all of them get the same C function in the CPython implementation. Whether the object is an instance of a class with the __call__ method, a function (the object itself), or an inline object, all calls (except for optimized special cases) go to the PyObject_Call function. This function C gets the type of the object from the ob_type field of the ob_type struct object, and then from the type (another PyObject struct) it gets the tp_call field, which is a pointer to the function. If tp_call not NULL , it calls this, with the args and kwargs structures that were also passed to PyObject_Call .

When the class defines the __call__ method, which sets the tp_call field tp_call .

This article details all of this: Python internals: how callables work . It even lists and explains the entire PyObject_Call function, which is not very large. If you want to see this function in its natural habitat, it is located in Objects / abstract.c in the CPython repository.

Also relevant is this stackoverflow Q&A: What is “callable” in Python? .

Answer 2

There is no infinite loop because the __call__ method is __call__ actually called ("called") for all of these situations. It is called directly when there is a functional call to the object that the __call__ method __call__ .

Creating an instance of the normal class Cls(...) and a regular function call f() are known cases that are handled directly. Usually there is no actual call to __call__() , so there are a finite number of calls to the __call__ method that may ever occur, even in complex cases with deep inheritance, metaclasses, etc.

Since there has been some debate as to whether the conceptual infinite loops really short circuit, let's look at the parsed bytecode. Consider the following code:

 def f(x): return x + 1 class Adder(object): def something(self, x): return x + 19 def __call__(self, x): return x + 1 def lotsacalls(y): u = f(1) a = Adder() z = u + a.something(y) return a(z * 10) 

Sorry, this is a bit complicated, because I want to show several instances of a short circuit - namely, the usual def functions, __init__ calls, ordinary methods and __call__ special methods. Now:

So, here is the range of times that if Python really were, really “walking the tree” of the conceptual __call__ invocations, it should reference the Function classes (and possibly the Method ) and call them __call__ ). This is not true. It uses a simple bytecode CALL_FUNCTION in all cases, a short circuit of the conceptual tree transition. Logically, you can imagine that there is a Function class that has a __call__ method that is called when a function is called (i.e., an instance of the Function class). But actually it is not. The compiler, bytecode interpreter, and other parts of the foundations of the C-language do not actually walk metaclass trees. They short out like crazy.

Answer 3

I have not checked the documentation, but from my tests it seems that __call__ not always called:

 def func1(*args, **kargs): print "func1 called", args, kargs def func2(*args, **kargs): print "func2 called", args, kargs func1.__call__ = func2 func1() # here is still called func1 class Cls: def __init__(*args, **kargs): print "init called", args, kargs def __call__(*args, **kargs): print "object called", args, kargs obj = Cls() # here is actually called __init__ obj() # here is called __call__ 

it prints

 func1 called () {} init called (<__main__.Cls instance at 0x0000000002A5ED88>,) {} object called (<__main__.Cls instance at 0x0000000002A5ED88>,) {}