Modifying a Python dict when iterating over it

Question

Say we have a Python d dictionary, and we repeat it like this:

 for k,v in d.iteritems(): del d[f(k)] # remove some item d[g(k)] = v # add a new item 

( f and g are just black box conversions.)

In other words, we are trying to add / remove elements in d , iterating over it using iteritems .

Is it well defined? Could you provide some recommendations to support your answer?

(It’s pretty obvious how to fix this if it’s broken, so that’s not the angle I get after.)

Answer 1

It is explicitly mentioned on the Python document page (for Python 2.7 ), which

Using iteritems() when adding or removing entries in a dictionary may raise a RuntimeError or iterate through all the elements.

Similarly for Python 3 .

The same is true for iter(d) , d.iterkeys() and d.itervalues() , and I will go as far as possible for for k, v in d.items(): (I can’t remember exactly what does for , but I won’t be surprised if the implementation is called iter(d) ).

Answer 2

Alex Martelli weighs here here .

It may not be safe to modify the container (e.g. dict) while cycling around the container. Thus, del d[f(k)] may be unsafe. As you know, a workaround is to use d.items() (to loop through an independent copy of the container) instead of d.iteritems() (which uses the same base container).

You can change the value in an existing dict index, but inserting values ​​into new indexes (for example, d[g(k)]=v ) may not work.

Answer 3

You cannot do this, at least with d.iteritems() . I tried this and Python fails with

 RuntimeError: dictionary changed size during iteration 

If you use d.items() , then it works.

In Python 3, d.items() is a kind of dictionary, for example d.iteritems() in Python 2. To do this, use d.copy().items() in Python 3. It will also allow us to iterate over a copy of the dictionary to avoid changing the data structure that we iterate.

Answer 4

The following code shows that this is incorrectly defined:

 def f(x): return x def g(x): return x+1 def h(x): return x+10 try: d = {1:"a", 2:"b", 3:"c"} for k, v in d.iteritems(): del d[f(k)] d[g(k)] = v+"x" print d except Exception as e: print "Exception:", e try: d = {1:"a", 2:"b", 3:"c"} for k, v in d.iteritems(): del d[f(k)] d[h(k)] = v+"x" print d except Exception as e: print "Exception:", e 

The first example calls g (k) and throws an exception (resized dictionary during iteration).

The second example calls h (k) and does not throw an exception, but outputs:

 {21: 'axx', 22: 'bxx', 23: 'cxx'} 

Which, looking at the code, seems wrong - I would expect something like:

 {11: 'ax', 12: 'bx', 13: 'cx'} 

Answer 5

I have a large dictionary containing Numpy arrays, so the thing is dict.copy (). keys () suggested by @ murgatroid99 was not possible (although it worked). Instead, I simply converted key_view to a list, and it worked perfectly (in Python 3.4):

 for item in list(dict_d.keys()): temp = dict_d.pop(item) dict_d['some_key'] = 1 # Some value 

I understand that this will not plunge into the philosophical sphere of internal Python processes, as the answers above, but it provides a practical solution to the stated problem.

Answer 6

I have the same problem and I used the following procedure to solve this problem.

A Python list can be iterated even if you modify it during iteration. therefore, for the following code, it will print 1 endlessly.

 for i in list: list.append(1) print 1 

Thus, using a list and a dict together, you can solve this problem.

 d_list=[] d_dict = {} for k in d_list: if d_dict[k] is not -1: d_dict[f(k)] = -1 # rather than deleting it mark it with -1 or other value to specify that it will be not considered further(deleted) d_dict[g(k)] = v # add a new item d_list.append(g(k)) 

Answer 7

Today I had a similar use case, but instead of just materializing the keys in the dictionary at the beginning of the loop, I wanted the changes in the dict to affect its iteration, which was orderly.

In the end, I created the following procedure, which can also be found in jaraco.itertools :

 def _mutable_iter(dict): """ Iterate over items in the dict, yielding the first one, but allowing it to be mutated during the process. >>> d = dict(a=1) >>> it = _mutable_iter(d) >>> next(it) ('a', 1) >>> d {} >>> d.update(b=2) >>> list(it) [('b', 2)] """ while dict: prev_key = next(iter(dict)) yield prev_key, dict.pop(prev_key) 

The document line illustrates usage. This function can be used instead of d.iteritems() above to get the desired effect.

Answer 8

Python 3 you should just:

 prefix = 'item_' t = {'f1': 'ffw', 'f2': 'fca'} t2 = dict() for k,v in t.items(): t2[k] = prefix + v 

or use:

 t2 = t1.copy() 

You should never modify the original dictionary, this leads to confusion, as well as potential errors or RunTimeErrors. Unless you add new key names to the dictionary.