Understanding list comprehension for aligning list of lists in python

Question

I found this understanding that works great for aligning a list of lists:

>>> list_of_lists = [(1,2,3),(2,3,4),(3,4,5)] >>> [item for sublist in list_of_lists for item in sublist] [1, 2, 3, 2, 3, 4, 3, 4, 5] 

I like it better than using itertools.chain() , but I just don't get it. I tried the surrounding parts with parentheses to see if I can reduce the complexity, but now I'm just more confused:

 >>> [(item for sublist in list_of_lists) for item in sublist] [<generator object <genexpr> at 0x7ff919fdfd20>, <generator object <genexpr> at 0x7ff919fdfd70>, <generator object <genexpr> at 0x7ff919fdfdc0>] >>> [item for sublist in (list_of_lists for item in sublist)] [5, 5, 5] 

I have a feeling that it’s hard for me to understand, because I don’t quite understand how generators work ... I mean, I thought I knew, but now I seriously doubt it. As I said, I like how compact this idiom is, and that’s exactly what I need, but I hate using code that I don’t understand.

Can someone explain what exactly is going on here?

Answer 1

Read the for loops as if they were nested from left to right. The expression on the left is the one that produces each value in the final list:

 for sublist in list_of_lists: for item in sublist: item # added to the list 

The lists also support if tags to filter which items are used; they can also be considered as nested operators in the same way as for loops.

By adding parentheses, you have changed the expression; everything in brackets is now the left expression to add:

 for item in sublist: (item for sublist in list_of_lists) # added to the list 

A for loop like this is a generator expression. It works just like comprehension of a list, except that it does not create a list. Items are instead manufactured on demand. You can specify a generator expression for the next value, then the next value, etc.

In this case, an existing sublist object must exist for this; outer contour no longer exceeds list_of_lists .

Your last attempt translates to:

 for sublist in (list_of_lists for item in sublist): item # aded to the list 

Here, list_of_lists is a loop element in a generator expression that circles over for item in sublist . Again, a sublist should already exist for this. The loop then adds the already existing item to the final output of the output.

In your case, apparently, a sublist is a list with three elements in it; Your final list is 3 items. item was tied to 5 , so you got 3 times 5 in your release.

Answer 2

List description

When I first started by understanding the list, I read it like English sentences, and I could easily understand them. For example,

 [item for sublist in list_of_lists for item in sublist] 

can be read as

 for each sublist in list_of_lists and for each item in sublist add item 

In addition, the filter part can be read as

 for each sublist in list_of_lists and for each item in sublist add item only if it is valid 

And the corresponding understanding will be

 [item for sublist in list_of_lists for item in sublist if valid(item)] 

Generators

They are similar to land mines that fire only when called with the next protocol. They are similar to functions, but until an exception is thrown or the end of the function is reached, they are not exhausted, and they can be called again and again. The important thing is that they maintain state between the previous call and the current one.

The difference between a generator and a function is that generators use the yield keyword to return the value to the caller. In the case of a generator expression, they are similar to list comprehension; a fist expression is the actual value that is “given”.

In this basic understanding, if we look at your expressions in a question,

 [(item for sublist in list_of_lists) for item in sublist] 

You mix list comprehension with generator expressions. It will be read as follows

 for each item in sublist add a generator expression which is defined as, for every sublist in list_of_lists yield item 

this is not what you had in mind. And since the generator expression is not iterated, the generator expression object is added to the list as is. Since they will not be evaluated without calling with the following protocol, they will not cause errors (if they are, if they do not have a syntax error). In this case, it will throw a runtime error because the sublist is not yet defined.

In addition, in the latter case

 [item for sublist in (list_of_lists for item in sublist)] 

 for each sublist in the generator expression, add item and the generator expression is defined as for each item in sublist yield list_of_lists. 

The for loop will iterate over any iterative using the following protocol. That way, the generator expression will be evaluated, and item will always be the last item in the iteration of the sublist , and you add this to the list. This will also cause a runtime error since the subscriptions are not yet defined.

Answer 3

Understanding the list works as follows:

 [<what i want> <for loops in the order you'd write them naturally>] 

In this case, <what I want> is each item in each sublist . To get these items, you simply iterate over the sublists in the source list and save / display each item in the sublist. Thus, the order of the for loops in a list comprehension is the same order you would use if you did not use a list comprehension. The only confusing part is that first <what I want> , and not inside the body of the last loop.