Recursive "all the way" through a list of lists - Python

Question

A difficult problem, given the list of lists (nested here no more than one level):

[['a', 'b'], 'c', ['d', 'e'], ['f', 'g'], 'h'] 

.. find all length lists, the same as the specified list, and contain all possible combinations of elements from the subscriptions, with exactly one element of this subscription in the same position as the original sublist (it should not even be put in words). That is, find this:

 ['a', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'g', 'h'] ['a', 'c', 'e', 'f', 'h'] ['a', 'c', 'e', 'g', 'h'] ['b', 'c', 'd', 'f', 'h'] ['b', 'c', 'd', 'g', 'h'] ['b', 'c', 'e', 'f', 'h'] ['b', 'c', 'e', 'g', 'h'] 

Now I have found a solution, but this is not satisfactory for me:

 def all_paths(s, acc=None, result=None): # not using usual "acc = acc or []" trick, because on the next recursive call "[] or []" would be # evaluated left to right and acc would point to SECOND [], which creates separate accumulator # for each call stack frame if acc is None: acc = [] if result is None: result = [] head, tail = s[0], s[1:] acc_copy = acc[:] for el in head: acc = acc_copy[:] acc.append(el) if tail: all_paths(tail, acc=acc, result=result) else: result.append(acc) return result 

As you can see, this is due to copying the list of TWICE drives, for a fairly obvious reason, that if the method of the .append () or .extend () method is called from the recursion stack, the battery will be modified, because it is passed by label (separation in the official lingui?).

I tried to build a solution that pop () s and append () s the relevant number of elements is disconnected from the battery, but cannot get it correctly:

 def all_p(s, acc=None, result=None, calldepth=0, seqlen=0): if acc is None: acc = [] if result is None: seqlen = len(s) result = [] head, tail = s[0], s[1:] for el in head: acc.append(el) if tail: all_p(tail, acc=acc, result=result, calldepth=calldepth+1, seqlen=seqlen) else: result.append(acc[:]) print acc for i in xrange(1+seqlen-calldepth): acc.pop() return result 

Result:

 ['a', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'g', 'h'] ['a', 'c', 'd', 'e', 'f', 'h'] ['a', 'c', 'd', 'e', 'g', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'g', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'e', 'f', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'e', 'g', 'h'] ['a', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'g', 'h'] ['a', 'c', 'd', 'e', 'f', 'h'] ['a', 'c', 'd', 'e', 'g', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'g', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'e', 'f', 'h'] ['a', 'c', 'd', 'e', 'b', 'c', 'd', 'e', 'g', 'h'] 

Obviously, this is due to the fact that as a depth recursion, there is a switch up and down the call chain, and I can not get the number pop () s to finish the list of batteries.

I understand that this is a little practical gain, since copying the list is O (n), while popping k elements from the list O (k), so there are not many differences, but I'm curious if this can be done.

(Background: I update the phonecode test code, http://page.mi.fu-berlin.de/prechelt/phonecode/ , and this is the part that finds all the words, but each fraction of the phone number can be displayed for several words, for example :

 ... '4824': ['fort', 'Torf'], '4021': ['fern'], '562': ['mir', 'Mix'] ... 

so I need to find all possible "paths" through the selected list of matching words and / or numbers corresponding to the specified phone number)

Questions, requests:

• Can I fix a version that does not copy the battery?

• is there a solution for this that uses the itertools module?

• Any other, better approach to this particular problem? how is a non-recursive solution, faster solution, less memory intensive?

Yes, I know this is a load problem, but if someone solves a non-empty part of them, I would be grateful. :-)

Answer 1

is there a solution for this using the itertools module?

Yes, and it's pretty simple with itertools.product() . This is enough for your specific example ...

 >>> import itertools >>> l = [['a', 'b'], 'c', ['d', 'e'], ['f', 'g'], 'h'] >>> for i in itertools.product(*l): print list(i) ['a', 'c', 'd', 'f', 'h'] ['a', 'c', 'd', 'g', 'h'] ['a', 'c', 'e', 'f', 'h'] ['a', 'c', 'e', 'g', 'h'] ['b', 'c', 'd', 'f', 'h'] ['b', 'c', 'd', 'g', 'h'] ['b', 'c', 'e', 'f', 'h'] ['b', 'c', 'e', 'g', 'h'] 

... but, as DSM pointed out in the comments, it only works because your example uses single-character strings that are objects of a sequence of length one. If this is always the case, you can express the list as follows:

 ['ab', 'c', 'de', 'fg', 'h'] 

However, in the general case, you probably want to make sure that all list items are sequences with something like this ...

 >>> l = [None, int, 0, 'abc', [1, 2, 3], ('a', 'b')] >>> for i in itertools.product(*[i if isinstance(i, (list, tuple)) else [i] for i in l]): print list(i) [None, <type 'int'>, 0, 'abc', 1, 'a'] [None, <type 'int'>, 0, 'abc', 1, 'b'] [None, <type 'int'>, 0, 'abc', 2, 'a'] [None, <type 'int'>, 0, 'abc', 2, 'b'] [None, <type 'int'>, 0, 'abc', 3, 'a'] [None, <type 'int'>, 0, 'abc', 3, 'b'] 

Any other, better approach to this particular problem? how is a non-recursive solution, faster solution, less memory intensive?

Any solution would probably have to use recursion in some way, if not on the stack and then on the heap.

Answer 2

This is a less Pythonesque, but possibly an unroll-able version:

 def all_paths(l): if 1 == len(l): return [ l ] down = all_paths(l[1:]) # We iterate only lists, not tuples, not strings if type(l[0]) in [ list ]: return [ [ ll ] + k for k in down for ll in l[0] ] return [ [ l[0] ] + k for k in down ] l = [['a', 'b'], 'c', ['d', 'e'], ['f', 'g'], 'h'] for i in all_paths(l): print i 

In this form, it is inferior in all respects to the itertools solution: twice as much time and more than four times affects memory:

 HEAP SUMMARY (itertools) in use at exit: 1,867,521 bytes in 537 blocks total heap usage: 12,904 allocs, 12,367 frees, 8,444,917 bytes allocated HEAP SUMMARY (lists) in use at exit: 1,853,779 bytes in 498 blocks total heap usage: 57,653 allocs, 57,155 frees, 9,272,129 bytes allocated 

Also, itertools function - at least in a naive way - gives way to itertools (three times slower) and doubles the memory allocation:

 HEAP SUMMARY: in use at exit: 1,853,779 bytes in 498 blocks total heap usage: 26,707 allocs, 26,209 frees, 8,812,153 bytes allocated def all_paths(l): m = len(l)-1 val = [ i for i in range(0,m) ] ndx = [ 0 for i in l ] top = [ len(i) if type(i) in [list] else 0 for i in l ] while(True): path = [ l[i] if top[i] == 0 else l[i][ndx[i]] for i in val ] #print path n = m ndx[n] = ndx[n] + 1 while (ndx[n] >= top[n]): ndx[n] = 0 n = n - 1 if (-1 == n): return ndx[n] = ndx[n] + 1 

(using enumerate or type(l[i]) leads to even worse performance).