Elements are missing when an iterator is used in list comprehension

Question

The following example shows different behavior depending on whether the right-most generator in the list comprehension is a list or an iterator. In particular, when using an iterator, fewer results are generated. I think this behavior is very surprising.

I am new to Python, so I think I am missing something obvious, but I would be grateful for the explanation.

>>> import itertools
>>> xs = range(0, 5)
>>> ys = range(0, 3)
>>> zs_l = [x for x in xs if not x in ys]
>>> zs_l
[3, 4]

# Validate the contents of the iterator, and create it again
>>> zs_i = itertools.ifilterfalse(lambda x: x in ys, xs)
>>> list(zs_i)
[3, 4]
>>> list(zs_i)
[]
>>> zs_i = itertools.ifilterfalse(lambda x: x in ys, xs)

>>> [(i,z) for i in [1,2] for z in zs_l]
[(1, 3), (1, 4), (2, 3), (2, 4)]
>>> [(i,z) for i in [1,2] for z in zs_i]
[(1, 3), (1, 4)]

Answer 1

Quote itertools.ifilterfalsedocs,

Make an iterator that filters the elements ...

Quoting python documentation for the term iterator ,

, . next() . , StopIteration. , next() StopIteration. __iter__(), , , . , . - (, list) , iter() for. , , .

.

>>> [(i,z) for i in [1,2] for z in zs_i]
[(1, 3), (1, 4)]

zs_i for. , for, , , StopIteration. , .

, range,

- (, list) , iter() for.

, for , , , .

Answer 2

itertools.ifilterfalse . , , yield, list , .

[(i,z) for i in [1,2] for z in zs_i]

zs_id i = 1. i = 2, zs_i .

Answer 3

This answer is in addition to other answers that explain the underlying mechanisms in more detail. If you want this to work, the generator needs to be recreated several times in understanding.

One way is to initialize a new generator for each pass of the nested for loop:

>>> [(i,z) for i in [1,2] for z in itertools.ifilterfalse(lambda x: x in ys, xs)]
[(1, 3), (1, 4), (2, 3), (2, 4)]