Iterators

Iterators provide an easy and comfortable way to provide data for a Sequences.

Iterators are special types of Methods that return a Sequence. Rather then executing all at once and returning a finished list, tough, bits of the iterator are run as needed, whenever a new ite for the sequence is requested.

Iterator Syntax

Iterators are declared like normal methods, but with three important distinctions:

1. Iterator methods must return a Sequence type.
2. They must me marked with the iterator Member Modifier.
3. Instead of returning a value, they pass back individual items of the sequence using yield.

Of course, any method can return a sequence, simply by declaring a sequence type as their result type. A normal method would need to handle the work of constructing the sequence itself – which can be easy when returning data from, say an Array or a list, but becomes more complicated when the data should be generated dynamically as the sequence is enumerated.

What the iterator syntax adds to the table is that it makes it easy to write linear code that returns one item of a sequence after the other, but allow that code to be executed non-linearly, piece by piece, if and when the sequence is accessed.

This is done via the yield statement), which is available only in iterators, and replaces result or the ability to exit with a value.

While the iterator is declared to return sequence of a specific type, inside it, each use of the yield statement will yield a single value of that type. For example

method NaturalNumbers: sequence of Integer; iterator;
begin
  for i := 1 to Int32.Max do
    yield i;
end;

...

for each n in NaturalNumbers do
  writeLn(n);

Under the Hood

When an iterator is called, the body of the iterator method is not actually executed to calculate the whole sequence. Instead, a new object is instantiated, representing the sequence. Once code starts to enumerate over the sequence, for example in a for each loop, the iterator's method body will be executed piece by piece in order to provide the sequence elements.

It will run until the first occurrence of yield, and that value will be returned as the first value of the sequence – and then stop. As the loop continues to iterate the sequence and asks for the next item, execution resumes at that position, until the next yield statement is encountered, and so on — until the method body reaches it's end, marking the end of the sequence.

Of course the above example is very simple (and could be easily reproduced without iterators, for example using a For Loop Expression. But iterators allow for arbitrary complexity in code, and as many yield statements as needed.

For example, one could easily imagine a complex parser that processes data, with many nested if/then Clauses or case Statements that dive deep into a data structure and yield new items for a sequence from many places.

Oxygene;'s iterator syntax allows such complex logic to be expressed as linear flow – leaving it to the compiler to unpack the code so that for each tem of the sequence, execution runs form one yield statement to the next, and no further.

method WeirdSequenceOfNumbers(aExtraWeird: Boolean): sequence of String; iterator;
begin
  yield 'One';
  yield 'Two';
  yield 'Three';
  for i := 4 to 20 do
    yield i.ToString;
  yield 'Twenty-1';
  if aExtraWeird then begin
    for i := 20 to 90 do
      yield (i*3).ToString;
    yield "Twohundred and seventyone";
  end
  else begin
    for i := 20 to 90 do 
      yield i;
  end;
end;

Delegating Iteration to a Sequence

The yield keyword also supports delegation of the iterator to a second sequence, as shown below:

var MoreValues := [3,4,5,6]; // array of Int32, can act as sequence

method SomeIntegers: sequence of Int32; iterator;
begin
  yield 1; // adds "1" to the sequence
  yield 2; // adds "2" to the sequence
  yield MoreValues; // adds "3" thru "6" to the sequence, from the array
  yield 7; // adds "7" to the sequence
end;

When the iterator reaches the relevant yield statement, subsequent items are retrieved from that sequence, until it ends. After that, it continues to the next local yield statement. One can think of it as shorthand for:

  yield 1; // adds "1" to the sequence
  yield 2; // adds "2" to the sequence
  for each i in MoreValues do
    yield i
  yield 7; // adds "7" to the sequence

Limitations for Code in Iterators

In general, almost all code constructs can be used in iterators, including complex loops and conditional statements.

Since yielding in an iterator returns back to the caller on every iteration, it's not allowed to yield from within a protected block such as a locking statement or a try/ finally block, as that would leave an unbalanced Exception Handling Stack.

In addition, while an iterator can use the exit Statement to prematurely finish it's execution in any place, it may not return a value as part of the exit. Similarly, the result Expression is not available.

Implementing Asynchronous Sequences

Iterators can implement an Asynchronous Sequences, simply by specifying the appropriate async sequence of X result type:

method SomeIntegers: async sequence of Int32; iterator;
begin
  for i: Integer := 0 to 25 do
    yield il;
end;

Asynchronous sequences can then be iterated via an * await for each Loop.

Static Methods, Visibility and Virtuality and Other Modifiers

Since iterators are methods, they can be made static, and visibility and virtuality apply just as for regular methods.

See Also

• Sequence Types
• yield Statements
• for each Loops
• for each Loop Expressions
• await for each Loop Statements