Creating dynamically named variables in a function in python 3 / Understanding exec / eval / locals in python 3

Question

First of all, let me say that I read a lot of threads with similar topics when creating dynamically named variables, but they mostly relate to Python 2 or suggest that you are working with classes. And yes, I read the behavior of the exec function in Python 2 and Python 3 .

I also know that creating dynamically named variables is a bad idea 99% of the time, and dictionaries are a way to get it, but I just want to know if this is possible and how exec and locals work in python 3.

I would like to show a little code example illustrating my question (fibonacci calculates fibonacci numbers, ListOfLetters provides ["A", "B", ...]):

def functionname(): for index, buchstabe in enumerate(ListOfLetters.create_list("A", "K"), 1): exec("{} = {}".format(buchstabe, fibonacci(index)) ) #A = 1, B = 1, C = 2, D = 3, E = 5,... print(index, buchstabe, eval(buchstabe)) #works nicely, eg prints "4 D 3" print(locals()) #pritns all locals: {'B': 1, 'A': 1, 'index': 11, 'C': 2, 'H': 21, 'K': 89, ... print(locals()['K']) #prints 89 as it should print(eval("K")) #prints 89 as it should print(K) #NameError: name 'K' is not defined 

So, at least in my understanding, there is some inconsistency in the behavior of locals() , since it contains variable names added by exec() , but variables are not available in this function.

It would be great if someone could explain this and say whether it is by design or if it is a real inconsistency in the language. Yes, I know that locals should not be changed, but I do not change it, I call exec() ...

Answer 1

If you don't know why something works in Python, it can often help put a behavior that confuses you into functions, and then parse it from Python bytecode using the dis module.

Let's start with a simpler version of your code:

 def foo(): exec("K = 89") print(K) 

If you run foo() , you will get the same exception that you see with your more complex function:

 >>> foo() Traceback (most recent call last): File "<pyshell#167>", line 1, in <module> foo() File "<pyshell#166>", line 3, in foo print(K) NameError: name 'K' is not defined 

Let's take it apart and see why:

 >>> import dis >>> dis.dis(foo) 2 0 LOAD_GLOBAL 0 (exec) 3 LOAD_CONST 1 ('K = 89') 6 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 9 POP_TOP 3 10 LOAD_GLOBAL 1 (print) 13 LOAD_GLOBAL 2 (K) 16 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 19 POP_TOP 20 LOAD_CONST 0 (None) 23 RETURN_VALUE 

The operation you need to pay attention to is the number indicated by "13". Here, the compiler processes the search for K in the last line of the function ( print(K) ). It uses the LOAD_GLOBAL , which fails because "K" is not a global variable name, but rather a value in our locals() dict (added by exec ).

What if we persuaded the compiler to see K as a local variable (giving it a value before running exec ), so that it would know that it would not look for a global variable that did not exist?

 def bar(): K = None exec("K = 89") print(K) 

This function will not give you an error if you run it, but you do not get the expected value:

 >>> bar() None 

Let's take it apart to understand why:

 >>> dis.dis(bar) 2 0 LOAD_CONST 0 (None) 3 STORE_FAST 0 (K) 3 6 LOAD_GLOBAL 0 (exec) 9 LOAD_CONST 1 ('K = 89') 12 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 15 POP_TOP 4 16 LOAD_GLOBAL 1 (print) 19 LOAD_FAST 0 (K) 22 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 25 POP_TOP 26 LOAD_CONST 0 (None) 29 RETURN_VALUE 

Pay attention to the operation codes used in "3" and "19". The Python compiler uses STORE_FAST and LOAD_FAST to put the value for the local variable K in slot 0 and later to return it. Using numbered slots is significantly faster than inserting and fetching values ​​from a dictionary like locals() , so the Python compiler does this for all local access variables in a function. You cannot overwrite a local variable in a slot by changing the dictionary returned by locals() (like exec , unless you pass it a dict for use in your namespace).

In fact, let's try the third version of our function, where we again look at locals when we have K , defined as a regular local variable:

 def baz(): K = None exec("K = 89") print(locals()) 

You won't see 89 in the output this time either!

 >>> baz() {"K": None} 

The reason you see the old K value in locals() is explained in the function documentation :

Refresh and return a dictionary representing the current local character table.

The slot in which the local variable K is stored has not been modified by the exec statement, which only modifies the locals() dict. When you call locals() again, Python "updates [s]" the dictionary with the value from the slot, replacing the value stored there with exec .

That's why the docs say:

Note: The contents of this dictionary should not be changed; changes may not affect the values ​​of local and free variables used by the interpreter.

Your call to exec modifies the locals() dict and you will learn how its changes are not always visible by your later code.

Answer 2

In the question exec / eval / locals

At least when implementing the CPython modification in the locals() dictionary, the names in the local scope are not actually changed, therefore it is intended for read-only use. You can change it, and you can see your changes in the dictionary object, but the actual local area does not change.

exec() takes two optional dictionary arguments, a global scope and a local scope. By default it is equal to globals() and locals() , but since changes to locals() are not "real" outside the dictionary, exec() only affects the "real" local area when globals() is locals() , i.e. in a module outside of any function. (Thus, in your case, this does not work, because it is inside the function area).

The “best” way to use exec() in this case is to pass into your own dictionary, and then work with the values ​​in that.

 def foo(): exec_scope = {} exec("y = 2", exec_scope) print(exec_scope['y']) foo() 

In this case, exec_scope used as the global and local scope for exec , and after exec it will contain {'y': 2, '__builtins__': __builtins__} (inline ones are inserted for you if not present)

If you want to access more global objects, you can do exec_scope = dict(globals()) .

Passing in different dictionaries of global and local coverage can lead to "interesting" behavior.

If you pass the same dictionary to successive exec or eval calls, then they have the same scope, so your eval worked (it implicitly used the locals() dictionary).

In dynamic variable names

If you specify a name from a string, what is wrong to get the value as a string (for example, what does the dictionary do)? In other words, why do you want to set locals()['K'] and then access K ? If K is in your source, this is not a really dynamic name ... hence dictionaries.