2.25. Generator

Generators allow you to declare a lambda that behaves like an iterator. For all intents and purposes, a generator is a lambda passed to an each or each_ref function.

Generator syntax is similar to lambda syntax:

generator ::= `generator` < type > $ ( ) block

Generator lambdas must have no arguments. It always returns boolean:

let gen <- generator<int>() <| $ {  // gen is iterator<int>
    for ( t in range(0,10) ) {
        yield t
    }
    return false                    // returning false stops iteration
}

The result type of a generator expression is an iterator (see Iterators).

Generators output iterator values via yield expressions. Similar to the return statement, move semantic yield <- is allowed:

return <- generator<TT> () <| $ {
    for ( w in src ) {
        yield <- invoke(blk,w)  // move invoke result
    }
    return false
}

Generators can output ref types. They can have a capture section:

unsafe {                                                // unsafe due to capture of src by reference
    var src = [1,2,3,4]
    var gen <- generator<int&> capture(ref(src)) () <| $ {      // capturing src by ref
        for ( w in src ) {
            yield w                                     // yield of int&
        }
        return false
    }
    for ( t in gen ) {
        t ++
    }
    print("src = {src}\n")  // will output [[2;3;4;5]]
}

Generators can have loops and other control structures:

let gen <- generator<int>() <| $ {
    var t = 0
    while ( t < 100 ) {
        if ( t == 10 ) {
            break
        }
        yield t ++
    }
    return false
}

let gen <- generator<int>() <| $ {
    for ( t in range(0,100) ) {
        if ( t >= 10 ) {
            continue
        }
        yield t
    }
    return false
}

Generators can have a finally expression on its blocks, with the exception of the if-then-else blocks:

let gen <- generator<int>() <| $ {
    for ( t in range(0,9) ) {
        yield t
    } finally {
        yield 9
    }
    return false
}

2.25.1. implementation details

In the following example:

var gen <- generator<int> () <| $ {
    for ( x in range(0,10) ) {
        if ( (x & 1)==0 ) {
            yield x
        }
    }
    return false
}

A lambda is generated with all captured variables:

struct _lambda_thismodule_8_8_1 {
    __lambda : function<(__this:_lambda_thismodule_8_8_1;_yield_8:int&):bool const> = @@_::_lambda_thismodule_8_8_1`function
    __finalize : function<(__this:_lambda_thismodule_8_8_1? -const):void> = @@_::_lambda_thismodule_8_8_1`finalizer
    __yield : int
    _loop_at_8 : bool
    x : int // captured constant
    _pvar_0_at_8 : void?
    _source_0_at_8 : iterator<int>
}

A lambda function is generated:

[GENERATOR]
def _lambda_thismodule_8_8_1`function ( var __this:_lambda_thismodule_8_8_1; var _yield_8:int& ) : bool const {
    goto __this.__yield
    label 0:
    __this._loop_at_8 = true
    __this._source_0_at_8 <- __::builtin`each(range(0,10))
    memzero(__this.x)
    __this._pvar_0_at_8 = reinterpret<void?> addr(__this.x)
    __this._loop_at_8 &&= _builtin_iterator_first(__this._source_0_at_8,__this._pvar_0_at_8,__context__)
    label 3: /*begin for at line 8*/
    if ( !__this._loop_at_8 ) {
            goto label 5
    }
    if ( !((__this.x & 1) == 0) ) {
            goto label 2
    }
    _yield_8 = __this.x
    __this.__yield = 1
    return /*yield*/ true
    label 1: /*yield at line 10*/
    label 2: /*end if at line 9*/
    label 4: /*continue for at line 8*/
    __this._loop_at_8 &&= _builtin_iterator_next(__this._source_0_at_8,__this._pvar_0_at_8,__context__)
    goto label 3
    label 5: /*end for at line 8*/
    _builtin_iterator_close(__this._source_0_at_8,__this._pvar_0_at_8,__context__)
    return false
}

Control flow statements are replaced with the label + goto equivalents. Generators always start with goto __this.yield. This effectively produces a finite state machine, with the yield variable holding current state index.

The yield expression is converted into a copy result and return value pair. A label is created to specify where to go to next time, after the yield:

_yield_8 = __this.x                 // produce next iterator value
__this.__yield = 1                  // label to go to next (1)
return /*yield*/ true               // return true to indicate, that iterator produced a value
label 1: /*yield at line 10*/       // next label marker (1)

Iterator initialization is replaced with the creation of the lambda:

var gen:iterator<int> <- each(new<lambda<(_yield_8:int&):bool const>> default<_lambda_thismodule_8_8_1>)