The fastest way to remove specific keys from a dict in Python

Question

I am looking for the fastest / most efficient way to remove specific keys in python python

Here are some options.

for k in somedict.keys(): if k.startswith("someprefix"): del somedict[k] 

or

 dict((k, v) for (k, v) in somedict.iteritems() if not k.startswith('someprefix')) 

Logically, the first snippet should be faster on smaller dicts, it does not create a copy of dict, but creates a list of all keys, however, double searching and rebuilding dict takes a lot of time. While the second is faster on large voice recorders, but requires 2x more memory. I tested my assumption in a little test.

Anything faster?

Answer 1

If the dict is large enough, it might make sense to generate a new new dict.

 dict((k, v) for (k, v) in somedict.iteritems() if not k.startswith('someprefix')) 

Answer 2

Not only del easier to understand, but it looks a bit faster than pop () :

 $ python -m timeit -s "d = {'f':1,'foo':2,'bar':3}" "for k in d.keys():" " if k.startswith('f'):" " del d[k]" 1000000 loops, best of 3: 0.733 usec per loop $ python -m timeit -s "d = {'f':1,'foo':2,'bar':3}" "for k in d.keys():" " if k.startswith('f'):" " d.pop(k)" 1000000 loops, best of 3: 0.742 usec per loop 

---

Edit: thanks to Alex Martelli for providing instructions on how to do this. I hope I haven’t gone anywhere.

First measure the time it takes to copy:

 $ python -m timeit -s "d = {'f':1,'foo':2,'bar':3}" "d1 = d.copy()" 1000000 loops, best of 3: 0.278 usec per loop 

Check parameter for copied dict:

 $ python -m timeit -s "d = {'f':1,'foo':2,'bar':3}" "d1 = d.copy()" "for k in d1.keys():" " if k.startswith('f'):" " del d1[k]" 100000 loops, best of 3: 1.95 usec per loop $ python -m timeit -s "d = {'f':1,'foo':2,'bar':3}" "d1 = d.copy()" "for k in d1.keys():" " if k.startswith('f'):" " d1.pop(k)" 100000 loops, best of 3: 2.15 usec per loop 

Subtracting the cost of copying, we get 1.872 usec for pop() and 1.672 for del .