The volume of lambda functions and their parameters?

Question

I need a callback function that is pretty much the same for a series of gui events. The function will behave somewhat differently, depending on what event caused this event. This seems to be a simple case, but I cannot understand this strange behavior of lambda functions.

So, I have the following simplified code:

def callback(msg): print msg #creating a list of function handles with an iterator funcList=[] for m in ('do', 're', 'mi'): funcList.append(lambda: callback(m)) for f in funcList: f() #create one at a time funcList=[] funcList.append(lambda: callback('do')) funcList.append(lambda: callback('re')) funcList.append(lambda: callback('mi')) for f in funcList: f() 

The output of this code is:

 mi mi mi do re mi 

I expected:

 do re mi do re mi 

Why is iterator usage confused?

I tried using deepcopy:

 import copy funcList=[] for m in ('do', 're', 'mi'): funcList.append(lambda: callback(copy.deepcopy(m))) for f in funcList: f() 

But this has the same problem.

Answer 1

The problem here is the variable m (link) taken from the environment. Only parameters are stored in the lambda area.

To solve this problem, you need to create another area for lambda:

 def callback(msg): print msg def callback_factory(m): return lambda: callback(m) funcList=[] for m in ('do', 're', 'mi'): funcList.append(callback_factory(m)) for f in funcList: f() 

In the above example, the lambda also uses the environment to find m , but this is the time callback_factory that is created once for each callback_factory call.

Or with functools.partial :

 from functools import partial def callback(msg): print msg funcList=[partial(callback, m) for m in ('do', 're', 'mi')] for f in funcList: f() 

Answer 2

When a lambda is created, it does not create a copy of the variables in the scope that it uses. It maintains a reference to the environment to subsequently view the value of the variable. There is only one m . He is attached to each cycle. After the loop, the variable m has the value 'mi' . Therefore, when you actually run the function that you created later, it will look for the value m in the environment that created it, and by then it will have the value 'mi' .

One common and idiomatic solution to this problem is to fix the value of m during lambda creation, using it as the default argument for an optional parameter. Usually you use a parameter with the same name, so you do not need to change the body of the code:

 for m in ('do', 're', 'mi'): funcList.append(lambda m=m: callback(m)) 

Answer 3

Python really uses links, of course, but that doesn't matter in this context.

When you define a lambda (or function, since this is the same behavior), it does not evaluate the lambda expression before the runtime:

 # defining that function is perfectly fine def broken(): print undefined_var broken() # but calling it will raise a NameError 

Even more surprising than your lambda example:

 i = 'bar' def foo(): print i foo() # bar i = 'banana' foo() # you would expect 'bar' here? well it prints 'banana' 

In short, think dynamically: nothing is evaluated before interpretation, so your code uses the last value of m.

When he searches for m in lambda execution, m is taken from the upper region, which means that, as others have pointed out; you can work around this problem by adding another area:

 def factory(x): return lambda: callback(x) for m in ('do', 're', 'mi'): funcList.append(factory(m)) 

Here, when a lambda is called, it looks at the lambda definition area for x. This x is a local variable defined in the body of the factory. Because of this, the value used when executing the lambda will be the value that was passed as a parameter during the factory call. And finish off!

As a note, I could define factory as factory (m) [replace x with m], the behavior will be the same. I used a different name for clarity :)

You may find that Andrej Bauer got similar problems with lambda. What is interesting about this blog is the comments in which you will learn more about closing python :)

Answer 4

Not directly related to the problem, but an invaluable piece of wisdom: Python objects from Fredrik Lundh.

Answer 5

Firstly, what you see is not a problem and is not related to a call by reference or by value.

The lambda syntax you define has no parameters, and thus the scope with the parameter m is external to the lambda function. That is why you see these results.

The Lambda syntax is not needed in your example, and you would prefer to use a simple function call:

 for m in ('do', 're', 'mi'): callback(m) 

Again, you have to be very accurate about the lambda parameters you use, and where exactly their area starts and ends.

As a side note regarding parameter passing. Parameters in python always refer to objects. To quote Alex Martelli:

A terminological problem may be caused by the fact that in python the value of the name is a reference to an object. So, you always pass a value (no implicit copying), and that value is always a reference. [...] Now, if you want to assign a name for this, for example, "by object link", "unreasonable" value "or something else, be my guest. An attempt to reuse terminology, which is more widely applied to languages ​​where "variables are fields" for a language where "variables are post-it tags", IMHO, with a greater likelihood of confusing than helping.

Answer 6

The variable m is fixed, so your lambda expression always sees its "current" value.

If you need to effectively capture a value at a point in time, write that the function takes the value you want as a parameter and returns a lambda expression. At this moment, the lambda will fix the value of the parameter, which will not change when the function is called several times:

 def callback(msg): print msg def createCallback(msg): return lambda: callback(msg) #creating a list of function handles with an iterator funcList=[] for m in ('do', 're', 'mi'): funcList.append(createCallback(m)) for f in funcList: f() 

Output:

 do re mi 

Answer 7

Yes, the scope problem, it binds to the outer m, regardless of whether you use a lambda or a local function. Use a functor instead:

 class Func1(object): def __init__(self, callback, message): self.callback = callback self.message = message def __call__(self): return self.callback(self.message) funcList.append(Func1(callback, m)) 

Answer 8

there are actually no variables in the classic sense in Python, just names tied to references to the applicable object. Even functions are a kind of object in Python, and lambdas do not make an exception to the rule :)

Answer 9

As a side note, map , though despised by some famous Python figure, forces a construct that prevents this trap.

 fs = map (lambda i: lambda: callback (i), ['do', 're', 'mi']) 

NB: the first lambda i acts like a factory in other answers.

Answer 10

soluiton to lambda more lambda

 In [0]: funcs = [(lambda j: (lambda: j))(i) for i in ('do', 're', 'mi')] In [1]: funcs Out[1]: [<function __main__.<lambda>>, <function __main__.<lambda>>, <function __main__.<lambda>>] In [2]: [f() for f in funcs] Out[2]: ['do', 're', 'mi'] 

external lambda used to bind the current value of i to j on

every time the external lambda is called, it makes an instance of the internal lambda with j bound to the current value of i as i value