__iter __ () implemented as a generator

Question

I have a subclass of an object that implements __ iter __ dynamic dispatch using a cache generator (I also have a method for iter cache invalidation) as follows:

 def __iter__(self): print("iter called") if self.__iter_cache is None: iter_seen = {} iter_cache = [] for name in self.__slots: value = self.__slots[name] iter_seen[name] = True item = (name, value) iter_cache.append(item) yield item for d in self.__dc_list: for name, value in iter(d): if name not in iter_seen: iter_seen[name] = True item = (name, value) iter_cache.append(item) yield item self.__iter_cache = iter_cache else: print("iter cache hit") for item in self.__iter_cache: yield item 

It seems to work ... Are there any bugs that I may not know about? Am I doing something funny?

Answer 1

This seems like a very fragile approach. It is enough to change any of __slots, __dc_list, __iter_cache during the active iteration to put the object in an inconsistent state.

You need to either prohibit changing the object during the iteration, or immediately create all elements of the cache and return a copy of the list.

Answer 2

container.__iter__() returns an iterator object. Iterators themselves must support the following two methods, which together form the iterator protocol:

 iterator.__iter__() 

Returns the iterator object itself.

 iterator.next() 

Returns the next item from the container.

What every generator has. Therefore, do not be afraid of any side effects.

Answer 3

It might be better to separate the iteration of the object from the caching of the return values. This will simplify the iteration process and allow you to easily control how caching is performed, as well as whether it is enabled or not, for example.

Another, perhaps important, consideration is the fact that your code will not predictably handle the situation where an object that repeats changes between successive method calls. One easy way to handle this is to completely populate the contents of the cache on the first call, and then simply yield what it contains for each call, and document the behavior.

Answer 4

What you do is really permissible, albeit strange. What is __slots or __dc_list ? As a rule, it is better to describe the contents of your object in the attribute name rather than its type (for example: self.users, not self.u_list).

You can use the LazyProperty decorator to simplify this.

Just decorate your method with @LazyProperty. It will be called for the first time, and the decorator will then replace the attribute with the results. The only requirement is that the value is repeatable; it does not depend on a volatile state. You also have this requirement in your current code, with your .__ iter_cache.

 def __iter__(self) return self.__iter @LazyProperty def __iter(self) def my_generator(): yield whatever return tuple(my_generator())