Python: Elegant way to double / multiple iterate over the same list

Question

I wrote some code, as shown below, to compare items with other items on the list. Is there a more elegant template for this kind of double iteration?

jump_item_iter = (j for j in items if some_cond) try: jump_item = jump_item_iter.next() except StopIteration: return for item in items: if jump_item is item: try: jump_item = jump_iter.next() except StopIteration: return # do lots of stuff with item and jump_item 

I don't think " except StopIteration " is very elegant

Edit:

To reliably make this clearer, I want to visit each item in the list and connect it to the next item further in the list (jump_item), which satisfies some_cond.

Answer 1

As far as I can see that any of the existing solutions work with a common single shot, possibly an endless iter ator , all of them seem to need an iter .

Here is the solution.

 def batch_by(condition, seq): it = iter(seq) batch = [it.next()] for jump_item in it: if condition(jump_item): for item in batch: yield item, jump_item batch = [] batch.append(jump_item) 

This will work easily with infinite iterators:

 from itertools import count, islice is_prime = lambda n: n == 2 or all(n % div for div in xrange(2,n)) print list(islice(batch_by(is_prime, count()), 100)) 

This will print the first 100 integers with the prime number that follows them.

Answer 2

I have no idea what compare() does, but in 80% of cases you can do this with a trivial dictionary or a pair of dictionaries. A list hint is a kind of linear search. If possible, a linear search should always be replaced with either a direct link (i.e. a Dict) or a tree (using the bisect module).

Answer 3

How about this?

 paired_values = [] for elmt in reversed(items): if <condition>: current_val = elmt try: paired_values.append(current_val) except NameError: # for the last elements of items that don't pass the condition pass paired_values.reverse() for (item, jump_item) in zip(items, paired_values): # zip() truncates to len(paired_values) # do lots of stuff 

If the first element of the elements matches, then it is used as jump_item. This is the only difference from your source code (and you may need this behavior).

Answer 4

The following iterator has time and memory:

 def jump_items(items): number_to_be_returned = 0 for elmt in items: if <condition(elmt)>: for i in range(number_to_be_returned): yield elmt number_to_be_returned = 1 else: number_to_be_returned += 1 for (item, jump_item) in zip(items, jump_items(items)): # do lots of stuff 

Note that you can indeed set the first number_to_be_returned to 1 ...

Answer 5

Write the function of the generator:

 def myIterator(someValue): yield (someValue[0], someValue[1]) for element1, element2 in myIterator(array): # do something with those elements. 

Answer 6

I have no idea what you are trying to do with this code. But I'm 99% sure that whatever it is, you can probably do it in 2 lines. I also get the feeling that the '==' operator should be the 'is' operator, otherwise what does the compare () function do? And what happens if the item returned from the second call to jump_iter.next is also equal to "item"? It seems that the algorithm will go wrong, as you compare the second, not the first.

Answer 7

You can put the whole iteration in a single try structure, so it would be clearer:

 jump_item_iter = (j for j in items if some_cond) try: jump_item = jump_item_iter.next() for item in items: if jump_item is item: jump_item = jump_iter.next() # do lots of stuff with item and jump_item except StopIteration: pass 

Answer 8

So, you want to compare pairs of elements in the same list, the second element of the pair must meet some condition. Usually, when you want to compare pairs in a list, use zip (or itertools.izip ):

 for item1, item2 in zip(items, items[1:]): compare(item1, item2) 

Find out how to set your some_cond in this :)

Answer 9

Are you basically trying to compare each item in the iterator with any other item in the source list?

In my opinion, this should be the case of using two cycles, instead of trying to fit it into one.

 filtered_items = (j for j in items if some_cond) for filtered in filtered_items: for item in items: if filtered != item: compare(filtered, item)
filtered_items = (j for j in items if some_cond) for filtered in filtered_items: for item in items: if filtered != item: compare(filtered, item) 

Answer 10

Here is one simple solution that might look a little cleaner:

 for i, item in enumerate(items): for next_item in items[i+1:]: if some_cond(next_item): break # do some stuff with both items 

The downside is that you check the condition for next_item several times. But you can easily optimize this:

 cond_items = [item if some_cond(item) else None for item in items] for i, item in enumerate(items): for next_item in cond_items[i+1:]: if next_item is not None: break # do some stuff with both items 

However, both solutions carry more overhead than the original solution from the question. And when you start using counters to get around this, I think it's better to use the iterator interface directly (as in the original solution).

Answer 11

My first answer was wrong, because I did not quite understand what you were trying to achieve. Therefore, if I understand correctly (this time, I hope), you want the main for item in items: "chase" after an iterator that filters out some elements. Well, not much that you can do, except maybe wrap it in a chase_iterator(iterable, some_cond) , which will make your main code a little more readable.

Perhaps a more readable approach would be a "cumulative approach" (if the order of comparison () does not matter), for example:

 others = [] for item in items: if some_cond(item): for other in others: compare(item, other) others = [] else: others.append(item) 

(man, I'm starting to hate Stack Overflow ... too exciting ...)

Answer 12

 for i in range( 0, len( items ) ): for j in range( i+1, len( items ) ): if some_cond: #do something #items[i] = item, items[j] = jump_item 

Answer 13

Using only iterators

 def(lst, some_cond): jump_item_iter = (j for j in lst if som_cond(j)) pairs = itertools.izip(lst, lst[1:]) for last in jump_item_iter: for start, start_next in itertools.takewhile(lambda pair: pair[0] < last, pairs): yield start, last pairs = itertools.chain([(start_next, 'dummy')], pairs) 

with input: range (10) and some_cond = lambda x: x% 2 gives [(0, 1), (1, 3), (2, 3), (3, 5), (4, 5), (5 , 7), (6, 7), (7, 9), (8, 9)] (same as your example)

Answer 14

Better yet, use itertools.groupby:

 def h(lst, cond): remain = lst for last in (l for l in lst if cond(l)): group = itertools.groupby(remain, key=lambda x: x < last) for start in group.next()[1]: yield start, last remain = list(group.next()[1]) 

Usage: lst = range (10) cond = lambda x: x% 2 print list (h (lst, cond))

will print

 [(0, 1), (1, 3), (2, 3), (3, 5), (4, 5), (5, 7), (6, 7), (7, 9), (8, 9)] 

Answer 15

It may already be too late, but what about:

 l = [j for j in items if some_cond] for item, jump_item in zip(l, l[1:]): # do lots of stuff with item and jump_item 

If l = [j for j in the range (10), if j% 2 == 0], then the iteration is completed: [(0, 2), (2, 4), (4, 6), (6, 8) ].

Answer 16

You can write the body of the loop as:

 import itertools, functools, operator for item in items: jump_item_iter = itertools.dropwhile(functools.partial(operator.is_, item), jump_item_iter) # do something with item and jump_item_iter 

dropwhile will return an iterator that skips all those that match the condition (here " is item ").

Answer 17

You can do something like:

 import itertools def matcher(iterable, compare): iterator= iter(iterable) while True: try: item= iterator.next() except StopIteration: break iterator, iterator2= itertools.tee(iterator) for item2 in iterator2: if compare(item, item2): yield item, item2 

but it is quite complex (and actually not very efficient), and it would be easier if you just did

 items= list(iterable) 

and then just write two loops over the items .

Obviously this will not work with infinite iterators, but your specification can only work with finite iterators.