Understanding python lists; list list compression?

Question

guys. I am trying to find the most elegant solution to the problem and wondering if python has something built in for what I'm trying to do.

What I do is this. I have a list A , and I have a function f that takes an element and returns a list. I can use list comprehension to convert everything to A like this:

 [f(a) for a in A] 

But this returns a list of lists,

 [a1,a2,a3] => [[b11,b12],[b21,b22],[b31,b32]] 

I really want a flattened list,

 [b11,b12,b21,b22,b31,b32] 

Now, in other languages; it is traditionally called flatmap in functional programming languages, and .Net calls it SelectMany . Does python have something like this? Is there a neat way to map a function over a list and smooth out the result?

The actual problem I'm trying to solve is the following; starting with the directory listing, find all the subdirectories. So;

 import os dirs = ["c:\\usr", "c:\\temp"] subs = [os.listdir(d) for d in dirs] print subs 

currentliy gives me a list of lists, but I really need a list.

Answer 1

You can have nested iterations in one list comprehension:

 [filename for path in dirs for filename in os.listdir(path)] 

Answer 2

 >>> listOfLists = [[1, 2],[3, 4, 5], [6]] >>> reduce(list.__add__, listOfLists) [1, 2, 3, 4, 5, 6] 

I suppose the itertools solution is more efficient than this, but it is very pythonic and it does not allow importing the library for just one list operation.

Answer 3

You can find a good answer in itertools recipes:

 def flatten(listOfLists): return list(chain.from_iterable(listOfLists)) 

(Note: Python 2.6+ required)

Answer 4

You can simply do it right:

 subs = [] for d in dirs: subs.extend(os.listdir(d)) 

Answer 5

Proposed question is flatmap . Some implementation options are suggested, but they may create unnecessary interim lists. Here is one implementation based on iterators.

 def flatmap(func, *iterable): return itertools.chain.from_iterable(map(func, *iterable)) In [148]: list(flatmap(os.listdir, ['c:/mfg','c:/Intel'])) Out[148]: ['SPEC.pdf', 'W7ADD64EN006.cdr', 'W7ADD64EN006.pdf', 'ExtremeGraphics', 'Logs'] 

In Python 2.x, use itertools.map instead of map .

Answer 6

You can combine lists using the usual addition operator:

 >>> [1, 2] + [3, 4] [1, 2, 3, 4] 

The built-in sum function will add numbers to the sequence and may optionally start with a specific value:

 >>> sum(xrange(10), 100) 145 

Combine the above to smooth the list of lists:

 >>> sum([[1, 2], [3, 4]], []) [1, 2, 3, 4] 

Now you can define your flatmap :

 >>> def flatmap(f, seq): ... return sum([f(s) for s in seq], []) ... >>> flatmap(range, [1,2,3]) [0, 0, 1, 0, 1, 2] 

Edit: I just saw criticism in the comments for another answer , and I think it is correct that Python will be useless to build and garbage collect many smaller lists with this solution. Therefore, the best that can be said about this is that it is very simple and concise if you are used to functional programming :-)

Answer 7

 import itertools x=[['b11','b12'],['b21','b22'],['b31']] y=list(itertools.chain(*x)) print y 

itertools will work from python2.3 and higher

Answer 8

 subs = [] map(subs.extend, (os.listdir(d) for d in dirs)) 

(but Ants answer is better, +1 for it)

Answer 9

You can try itertools.chain() , for example:

 import itertools import os dirs = ["c:\\usr", "c:\\temp"] subs = list(itertools.chain(*[os.listdir(d) for d in dirs])) print subs 

itertools.chain() returns an iterator, so go to list() .

Answer 10

Google brought me the following solution:

 def flatten(l): if isinstance(l,list): return sum(map(flatten,l)) else: return l 

Answer 11

 def flat_list(arr): send_back = [] for i in arr: if type(i) == list: send_back += flat_list(i) else: send_back.append(i) return send_back 

Answer 12

You can use pyxtension :

 from pyxtension.streams import stream stream([ [1,2,3], [4,5], [], [6] ]).flatMap() == range(7) 

Answer 13

 If listA=[list1,list2,list3] flattened_list=reduce(lambda x,y:x+y,listA) 

It will do.