Overloaded __iter__ bypasses on output from dict

Question

Trying to create a custom case insensitive dictionary, I received the following uncomfortable and (from my point of view) unexpected actions. If you deduce a class from a dict , the overloaded functions __iter__ , keys , values ignored when accessing the dict . I shortened it to the following test case:

 import collections class Dict(dict): def __init__(self): super(Dict, self).__init__(x = 1) def __getitem__(self, key): return 2 def values(self): return 3 def __iter__(self): yield 'y' def keys(self): return 'z' if hasattr(collections.MutableMapping, 'items'): items = collections.MutableMapping.items if hasattr(collections.MutableMapping, 'iteritems'): iteritems = collections.MutableMapping.iteritems d = Dict() print(dict(d)) # {'x': 1} print(dict(d.items())) # {'y': 2} 

The values ​​for keys , values and __iter__ , __getitem__ are incompatible only to demonstrate which methods are actually called.

The documentation for dict.__init__ states:

If a positional argument is given and it is the object of comparison, the dictionary is created with the same key-value pairs as the display object. Otherwise, the positional argument must be an iterator object.

I suppose this has something to do with the first sentence, and possibly with optimizations for the built-in dictionaries.

Why is the dict(d) call not used in any of the keys , __iter__ ? Is it possible to overload the "mapping" in some way to force the dict constructor to use my presentation of key-value pairs?

Why did I use this? For case insensitive but containing a dictionary, I wanted:

• store (lowercase => (original_case, value)) inside while it is displayed as (any_case => value).
• derives from a dict for working with some external library code that uses isinstance checks
• don't use 2 dictionary searches: lower_case => original_case and then the value of original_case => (this is the solution I'm doing now)

If you are interested in the application case: here is the corresponding branch

Answer 1

In the dictobject.c file , you see on line 1795ff. corresponding code:

 static int dict_update_common(PyObject *self, PyObject *args, PyObject *kwds, char *methname) { PyObject *arg = NULL; int result = 0; if (!PyArg_UnpackTuple(args, methname, 0, 1, &arg)) result = -1; else if (arg != NULL) { _Py_IDENTIFIER(keys); if (_PyObject_HasAttrId(arg, &PyId_keys)) result = PyDict_Merge(self, arg, 1); else result = PyDict_MergeFromSeq2(self, arg, 1); } if (result == 0 && kwds != NULL) { if (PyArg_ValidateKeywordArguments(kwds)) result = PyDict_Merge(self, kwds, 1); else result = -1; } return result; } 

This tells us that if an object has a keys attribute, the code being called is a simple merge. The code given there (l. 1915 ff.) Makes a distinction between real dicts and other objects. In the case of real dicts, elements are read using PyDict_GetItem() , which is the "innermost interface" for the object and does not require the use of any user methods.

Therefore, instead of inheriting from dict you should use the UserDict module.

Answer 2

Is there any way to overload the "mapping" to force the dict constructor to use my presentation of key-value pairs?

Not.

Being an integral type, redefinition of dict semantics will undoubtedly cause a direct break elsewhere.

You have a library in which you cannot override the dict behavior inside, this is tough, but overriding language primitives is not the answer. You would probably be uncomfortable if someone tied up with the commutative property of an integer addition behind your back; why they can’t.

And regarding your comment, “ UserDict (correctly) gives False in isinstance(d, dict) checks”, of course, he does, because it is not a dict and dict has very specific invariants that UserDict cannot guarantee.