Nim: How to wrap / infer an iterator from another iterator?

Question

Suppose we have some existingIterator that iterates over elements of an arbitrary type T Now I want to get a new iterator with existingIterator with modified behavior. Think of examples like:

• Limiting the length of the original iterator, e.g. existingIterator.take(n) .
• Display elements, e.g. existingIterator.map(modifier)
• Filtering specific elements, for example existingIterator.filter(predicate) .

In all these cases, I just want to create another iterator so that I can do something like this:

 for x in existingIterator.filter(something) .map(modifier) .take(10): ... 

My common problem: how can I write a generic iterator or template that accepts an existing iterator and returns a modified iterator?

The next question will be why such essential functionality is not available in the standard library - maybe I'm missing something?

---

Here is what I tried:

Attempt 1

Take the take(n) function as an example. My first approach was to use a regular general iterator :

 iterator infinite(): int {.closure.} = var i = 0 while true: yield i inc i iterator take[T](it: iterator (): T, numToTake: int): T {.closure.} = var i = 0 for x in it(): if i < numToTake: yield x inc i for x in infinite.take(10): echo x 

This compiler, but, unfortunately, it does not work: (1) the elements do not repeat correctly (are they all zero, maybe an error? ), (2) it looks like my program is stuck in an infinite loop and (3) it only works for closing iterators, which means that I cannot wrap arbitrary iterators.

Attempt 2

The restriction on closure iterators suggests that this problem actually requires a solution to the pattern.

 template take[T](it: iterator(): T, numToTake: int): expr {.immediate.} = var i = 0 iterator tmp(): type(it()) = for item in it: if i < numToTake: yield item inc i tmp 

It almost seems to work (i.e. compiling a template). However, if I now call for x in infinite.take(10) , I get:

 `Error: type mismatch: got (iterator (): int{.closure, gcsafe, locks: 0.})` 

I tried adding () to actually “call” the iterator, but it still doesn't work. So, it comes to the question: how do I create / return an iterator from a template?

Answer 1

The challenge lies in

 for x in infinite.take(10): echo x 

Or, more precisely, the call to infinite.take(10) , which we can also write as take(infinite, 10) . Unlike Sather , Nim has no once arguments for its iterators, so there is no way to distinguish between arguments that should be evaluated once per loop and arguments that should be evaluated once per iteration of the loop.

If you pass a closing iterator as an argument to another closing iterator, this means that a new instance of the infinite iterator with a new environment is created every time you go through the loop. This will cause infinite start from scratch again and again.

Inline iterators usually evaluate their arguments only once per loop (and in most cases this is the expected behavior). Closing iterators must undergo a transformation of their body into a state machine, which changes the way they are called. They can also be used in different ways: in particular, closing iterators can have several call sites, unlike built-in iterators; for example let iter = ...; iter(someArgument); iter(someOtherArgument) let iter = ...; iter(someArgument); iter(someOtherArgument) let iter = ...; iter(someArgument); iter(someOtherArgument) . As a result, I’m not sure if we are looking at the error or the alleged behavior here.

You can fix this by not directly passing infinite to take , but using let first. There is also an error in your take code that the loop does not end, which you also need to fix. The resulting code will look something like this:

 iterator infinite(): int {.closure.} = var i = 0 while true: yield i inc i iterator take[T](it: iterator (): T, numToTake: int): T {.closure.} = var i = 0 for x in it(): if i >= numToTake: break yield x inc i let inf = infinite for x in inf.take(10): echo x 

If you want to parameterize infinite , this can be done by transferring the iterator to a template or proc, for example:

 template infiniteFrom(x: int): (iterator (): int) = (iterator (): int = var i = x while true: yield i inc i) ... let inf = infiniteFrom(1) for x in inf.take(10): echo x 

Answer 2

I also tried adding functional methods to Nim, and I ended up wrapping everything in functions. Please see http://forum.nim-lang.org/t/1230 So you can assign a variable to an iterator before iterating through it with.