Close function against class to be called

Question

In many cases, there are two implementation options: closure and call class. For example,

class F: def __init__(self, op): self.op = op def __call__(self, arg1, arg2): if (self.op == 'mult'): return arg1 * arg2 if (self.op == 'add'): return arg1 + arg2 raise InvalidOp(op) f = F('add') 

or

 def F(op): if op == 'or': def f_(arg1, arg2): return arg1 | arg2 return f_ if op == 'and': def g_(arg1, arg2): return arg1 & arg2 return g_ raise InvalidOp(op) f = F('add') 

What factors should be considered when choosing, in any direction?

I can think of two:

• It seems that closing will always have better performance (can't think of a counterexample).

• I think there are times when a closure cannot do the job (for example, if its state changes over time).

Am I right in them? What else can I add?

Answer 1

Closing is faster. Classes are more flexible (i.e. more methods available than just __call __).

Answer 2

I understand that this is an older publication, but one of the factors that I have not seen is that in Python (pre-nonlocal) you cannot change the local variable contained in the linking environment. (In your example, such a modification is not important, but technically speaking, the inability to modify such a variable means that it is not a true closure.)

For example, the following code does not work:

 def counter(): i = 0 def f(): i += 1 return i return f c = counter() c() 

Calling c above will raise an UnboundLocalError exception.

This can be easily circumvented using mutable, for example a dictionary:

 def counter(): d = {'i': 0} def f(): d['i'] += 1 return d['i'] return f c = counter() c() # 1 c() # 2 

but, of course, this is just a workaround.

Answer 3

I find the class approach easier to understand at a glance and therefore more convenient to maintain. Since this is one of the prerequisites for good Python code, I believe that under equal conditions, it is better to use a class rather than a nested function. This is one of the cases where the flexible nature of Python causes the language to break "there must be one, and preferably only one, obvious way to do something" is a predicate for coding in Python.

The difference in performance of each of the parties should be insignificant - and if you have code on which performance is important at this level, you certainly should profile it and optimize the corresponding parts, perhaps by rewriting part of your code as your own code.

But yes, if there was a closed loop using state variables, evaluating closure variables should be slightly faster than evaluating class attributes. Of course, this could be overcome by simply inserting a string of type op = self.op inside the class method before entering the loop, making access to the variable inside the loop for the local variable - this will avoid searching for the attribute, and for each access. Again, the performance differences should be minor, and you have a more serious problem if you need this little extra performance and coding in Python.

Answer 4

Please note that due to an error previously discovered in my test code, my initial answer was incorrect. Below is a revised version.

I made a small program for measuring runtime and memory consumption. I created the following called class and closure:

 class CallMe: def __init__(self, context): self.context = context def __call__(self, *args, **kwargs): return self.context(*args, **kwargs) def call_me(func): return lambda *args, **kwargs: func(*args, **kwargs) 

I am confined to simple functions that take a different number of arguments ( math.sqrt() with 1 argument, math.pow() with 2 and max() with 12).

I used CPython 2.7.10 and 3.4.3+ for Linux x64. I could only do memory profiling in Python 2. The source code I used is available here .

My findings:

• Closures are faster than equivalent classes that cause a call: about 3 times faster in Python 2, but only in Python 3 1.5 times faster. The narrowing is due to the fact that closing has become slower and called classes are slower.
• Closures take up less memory than equivalent classes to be called: approximately 2/3 of the memory (tested only in Python 2).
• Although not part of the original question, it is interesting to note that the overhead of execution time for calls made using closure is about the same as calling math.pow() , while it doubles through the called class.

These are very rough estimates, and they may vary depending on the hardware, operating system, and function that you are comparing. However, this gives you an idea of ​​the effect of using each type of callable.

Therefore, it supports (unlike what I wrote earlier) that the accepted answer given by @RaymondHettinger is correct, and closure should be preferred for indirect calls, at least until it interferes with readability , Also thanks to @AXO for pointing out the error in my source code.

Answer 5

I would rewrite the class example with something like:

 class F(object): __slots__ = ('__call__') def __init__(self, op): if op == 'mult': self.__call__ = lambda a, b: a * b elif op == 'add': self.__call__ = lambda a, b: a + b else: raise InvalidOp(op) 

This gives 0.40 usec / pass (function 0.31, so 29% slower) on my machine with Python 3.2.2. Without using object as a base class, it gives 0.65 μs / skip (i.e., 55% slower than object ). And for some reason, code with op checking in __call__ gives almost the same results as if it were done in __init__ . With object as the base and check inside __call__ , 0.61 μs / pass is given.

The reason you use classes can be polymorphism.

 class UserFunctions(object): __slots__ = ('__call__') def __init__(self, name): f = getattr(self, '_func_' + name, None) if f is None: raise InvalidOp(name) else: self.__call__ = f class MyOps(UserFunctions): @classmethod def _func_mult(cls, a, b): return a * b @classmethod def _func_add(cls, a, b): return a + b