8.5.21. Macro Tutorial 21: Building a program from nodes

compile and compile_file (Compiling and Running Programs at Runtime) turn source text into a program. make_program skips the text. The host script builds each module out of AST nodes, and the compiler does the rest - inference, optimization, simulation - the same way it does for a parsed file.

Three functions from the ast module do the work:

make_program(cop) $ { ... } opens one compile. It has its own module group and loads builtin.das into it, the way compile_file does.
make_module(name) $(mod) { ... } builds a named module in the block and compiles it as soon as the block ends, like a required file. It returns the module, so the next modules can require it.
make_main_module() $(mod) { ... } builds the program’s own module and returns the finished program, ready for simulate.

Nothing here is a compile-time macro. The builders are ordinary functions that run while the host script runs.

8.5.21.1. The builders

The module file holds three functions that make Function nodes. make_greet_function builds greet one node at a time: a string builder, a call to print, a block, and the function with its argument. make_main_function and make_broken_function quote their bodies with qmacro_function, which is shorter and gives the same kind of nodes.

A function that the host calls by name must be exported, so main sets flags.exports. Without it, the program drops main as unused.

Full source: build_program_mod.das

options gen2

module build_program_mod public

require daslib/ast_boost public
require daslib/templates_boost public

def public make_greet_function() : FunctionPtr {
    var greeting = new ExprStringBuilder(at = LineInfo())
    greeting.elements |> emplace_new(new ExprConstString(at = LineInfo(), value := "hello, "))
    greeting.elements |> emplace_new(new ExprVar(at = LineInfo(), name := "name"))
    greeting.elements |> emplace_new(new ExprConstString(at = LineInfo(), value := "!\n"))
    var call = new ExprCall(at = LineInfo(), name := "print")
    call.arguments |> emplace(greeting)
    var body = new ExprBlock(at = LineInfo())
    body.list |> emplace(call)
    var fn = new Function(at = LineInfo(), atDecl = LineInfo(), name := "greet",
        result = new TypeDecl(at = LineInfo(), baseType = Type.tVoid))
    fn.arguments |> emplace_new(new Variable(at = LineInfo(), name := "name",
        _type = new TypeDecl(at = LineInfo(), baseType = Type.tString)))
    fn.body = body
    return fn
}

def public make_main_function() : FunctionPtr {
    var fn = qmacro_function("main") $() {
        greet("world")
        print("sqrt(2) = {sqrt(2.0)}\n")
    }
    fn.flags.exports = true
    return fn
}

def public make_broken_function() : FunctionPtr {
    return qmacro_function("broken") $() {
        greet(42)
    }
}

8.5.21.2. A program that runs

The greeter module is compiled when its block ends, so the main module can require it. math is a C++ module: get_module finds it, and the same add_module_require puts it in reach of the main module. add_function takes the function by reference, so each one goes into a variable first.

Full source: 21_build_program.das

options multiple_contexts

require daslib/debugger
require build_program_mod

def print_errors(program : ProgramPtr) {
    for (err in program.errors) {
        print("  error: {err.what}\n")
    }
}

def build_and_run() {
    using() $(var cop : CodeOfPolicies) {
        cop.threadlock_context = true
        make_program(cop) $ {
            let greeter = make_module("greeter") $(var mod) {
                var greet = make_greet_function()
                mod |> add_function(greet)
            }
            var inscope program <- make_main_module() $(var mmain) {
                mmain |> add_module_require(greeter, false)
                mmain |> add_module_require(get_module("math"), false)
                var main_fn = make_main_function()
                mmain |> add_function(main_fn)
            }
            if (program.failed) {
                print_errors(program)
                return
            }
            simulate(program) $(sok; context; serrors) {
                if (!sok) {
                    print("simulate error: {serrors}\n")
                    return
                }
                unsafe(invoke_in_context(context, "main"))
            }
        }
    }
}

The program is simulated and run inside make_program. Its modules live in the scope’s module group, and they are freed when the scope ends. A program that is still referenced when the block returns makes make_program panic. threadlock_context is set because invoke_in_context locks the program’s context.

8.5.21.3. A program that fails to compile

broken passes an int to greet(name : string). Inference of the main module rejects it. The program comes back with failed set and its errors in program.errors - the same errors a parsed file would report.

def build_broken() {
    using() $(var cop : CodeOfPolicies) {
        make_program(cop) $ {
            let greeter = make_module("greeter") $(var mod) {
                var greet = make_greet_function()
                mod |> add_function(greet)
            }
            var inscope program <- make_main_module() $(var mmain) {
                mmain |> add_module_require(greeter, false)
                var broken = make_broken_function()
                mmain |> add_function(broken)
            }
            if (program.failed) {
                print("compile failed, as expected\n")
                print_errors(program)
            }
        }
    }
}

When a module built with make_module fails, make_module returns null. The next make_main_module returns the failed program without running its block, so all the errors are checked in one place.

8.5.21.4. What happens underneath

A built program goes through the same steps as compile_file. Only the front end differs: a block fills each module where the parser would.

make_program loads the default modules into its module group.
make_module makes a program for one module, runs the block, then runs the same inference and finalization a required file gets. Then it moves the finished module into the group.
make_main_module builds the program’s own module in the same way, then gives it the steps an executable gets: unused symbols are removed and the stack is allocated.

Each module owns the nodes its block creates. While the block runs, the module’s own gc root is the active one, so every node the block allocates lands there. When the module ends, the nodes the module reaches are kept and the rest are freed. So build every node inside the block of the module that uses it: a function built anywhere else fails the build.

8.5.21.5. Output

=== greeter + main ===
hello, world!
sqrt(2) = 1.4142135

=== a type error ===
compile failed, as expected
  error: no matching functions or generics: greet(int const)