5.4.11. Macro Tutorial 11: Reader Macros

Previous tutorials intercepted calls, functions, structures, blocks, variants, for-loops, and lambda captures. Reader macros go further — they embed entirely custom syntax inside daslang source code.

[reader_macro(name="X")] registers a class that extends AstReaderMacro. Reader macros are invoked with the syntax %X~ character_sequence %%. The compiler calls three methods:

accept(prog, mod, expr, ch, info) → bool

Called per character during parsing. ch is the current character; expr.sequence accumulates the text. Return true to keep reading, false to stop (typically when %% is found).

visit(prog, mod, expr) → ExpressionPtr

Called during type inference on the ExprReader node. Must return an AST expression that replaces the reader expression. This is the visit pattern — used when the macro appears as an expression and produces an AST value.

suffix(prog, mod, expr, info, outLine&, outFile?&) → string

Called immediately after accept() during parsing. Returns a string of daslang source code that is injected back into the parser’s input stream. This is the suffix pattern — used when the macro appears at module level and generates top-level daslang declarations (functions, structs, etc.). The ExprReader node is discarded.

Note

The two patterns serve different contexts:

• Visit pattern: the reader macro appears in an expression position (e.g., var x = %csv~ ... %%). visit() builds the resulting AST node. suffix() is not used.

• Suffix pattern: the reader macro appears at module level as a standalone statement (not assigned to a variable). suffix() returns daslang source text that the parser re-parses. visit() is never called because the parser discards the ExprReader node.

All reader macros share the same accept() idiom for collecting characters. The choice between visit and suffix determines where and how the macro produces output.

5.4.11.1. Motivation

Embedding domain-specific notations — CSV data, regular expressions, JSON literals, template engines — is a common need. Reader macros let you write these in their native syntax and transform them at compile time into efficient daslang code, without runtime parsing overhead.

This tutorial builds both patterns:

• %csv~ — a visit reader macro that parses CSV text at compile time into a string array (constant embedded in the AST)

• %basic~ — a suffix reader macro that transpiles a toy BASIC program into a daslang function definition

Note

Macros cannot be used in the module that defines them. This tutorial has two source files: a module file containing the macro definitions and a usage file that requires the module.

5.4.11.2. The module file

reader_macro_mod.das defines two reader macros.

5.4.11.2.1. The accept() idiom

Both macros share the same standard accept() implementation — the most common pattern in the standard library:

def override accept(prog : ProgramPtr; mod : Module?;
        var expr : ExprReader?; ch : int; info : LineInfo) : bool {
    if (ch != '\r') {               // skip carriage returns
        append(expr.sequence, ch)    // accumulate in expr.sequence
    }
    if (ends_with(expr.sequence, "%%")) {
        let len = length(expr.sequence)
        resize(expr.sequence, len - 2)   // strip the %%
        return false                     // stop reading
    } else {
        return true                      // keep reading
    }
}

Characters are appended to expr.sequence one at a time. When the %% terminator is detected, it is stripped and accept() returns false to signal the end of the character sequence.

5.4.11.2.2. CsvReader — visit pattern

CsvReader is registered with [reader_macro(name=csv)]. Its visit() method splits the collected sequence by commas, trims each value, and uses convert_to_expression() from daslib/ast_boost to embed the resulting string array in the AST:

def override visit(prog : ProgramPtr; mod : Module?;
        expr : smart_ptr<ExprReader>) : ExpressionPtr {
    if (is_in_completion()) {
        return <- default<ExpressionPtr>
    }
    let seq = string(expr.sequence)
    var items <- split(seq, ",")
    for (i in range(length(items))) {
        items[i] = strip(items[i])
    }
    return <- convert_to_expression(items, expr.at)
}

convert_to_expression takes any daslang value and converts it into AST nodes — here turning an array<string> into the equivalent of an array literal. This is the same utility used by daslib/json_boost to embed parsed JSON and by daslib/regex_boost to embed compiled regex objects.

5.4.11.2.3. BasicReader — suffix pattern

BasicReader is registered with [reader_macro(name=basic)]. It overrides suffix() instead of visit(). The method parses a tiny BASIC dialect and returns the equivalent daslang source code:

def override suffix(prog : ProgramPtr; mod : Module?;
        var expr : ExprReader?; info : LineInfo;
        var outLine : int&; var outFile : FileInfo?&) : string {
    let seq = string(expr.sequence)
    var lines <- split(seq, "\n")
    var func_name = "basic_program"
    var stmts : array<string>
    for (line in lines) {
        let trimmed = strip(line)
        if (empty(trimmed)) { continue }
        if (starts_with(trimmed, "DEF ")) {
            func_name = strip(slice(trimmed, 4))
            continue
        }
        // parse: NUMBER COMMAND args
        let sp1 = find(trimmed, " ")
        if (sp1 < 0) { continue }
        let after_num = strip(slice(trimmed, sp1 + 1))
        if (starts_with(after_num, "PRINT ")) {
            let arg = strip(slice(after_num, 6))
            if (starts_with(arg, "\"")) {
                let inner = slice(arg, 1, length(arg) - 1)
                stmts |> push("print(\"{inner}\\n\")")
            } else {
                stmts |> push("print(\"\{{arg}\}\\n\")")
            }
        } elif (starts_with(after_num, "LET ")) {
            stmts |> push("var {strip(slice(after_num, 4))}")
        }
    }
    var result = "def {func_name}() \{\n"
    for (stmt in stmts) {
        result += "    {stmt}\n"
    }
    result += "\}\n"
    return result
}

The returned string is valid gen2 daslang code. The parser receives this text and parses it as a normal function definition at module level.

Note

The suffix must produce gen2 syntax (with braces) if the source file uses options gen2. String escaping requires care: \{ and \} produce literal braces (avoiding string interpolation), while {func_name} interpolates the variable.

5.4.11.3. The usage file

11_reader_macro.das demonstrates both patterns.

Section 1 — CSV reader (visit pattern):

var data <- %csv~ Alice, 30, New York %%
print("  items ({length(data)}):\n")
for (item in data) {
    print("    '{item}'\n")
}

Output:

--- Section 1: CSV reader (visit pattern) ---
  items (3):
    'Alice'
    '30'
    'New York'
  colors (3):
    'red'
    'green'
    'blue'

The %csv~ expression evaluates to an array<string> — the CSV values are parsed and embedded at compile time, not at runtime.

Section 2 — BASIC transpiler (suffix pattern):

%basic~
DEF basic_hello
10 PRINT "Hello from BASIC"
20 LET x = 42
30 PRINT x
%%

This appears at module level (not inside a function). The suffix generates a function basic_hello() that the rest of the file can call:

[export]
def main() {
    basic_hello()
}

Output:

--- Section 2: BASIC transpiler (suffix pattern) ---
Hello from BASIC
42

The generated function is indistinguishable from hand-written code — it participates in type checking, AOT compilation, and all other compiler phases normally.

5.4.11.4. Visit vs Suffix

Aspect	Visit pattern	Suffix pattern
Override	visit()	suffix()
When called	Type inference	Parsing (after accept)
Returns	AST expression	daslang source text
Usage context	Expression position	Module level
ExprReader fate	Replaced by visit result	Discarded by parser
Examples (stdlib)	regex, json, stringify	spoof_instance

5.4.11.5. Real-world usage

The standard library includes several reader macros:

• daslib/regex_boost.das — RegexReader (visit): compiles a regular expression at parse time and embeds the compiled Regex struct directly in the AST. Usage: %regex~pattern%%

• daslib/json_boost.das — JsonReader (visit): parses JSON at compile time and embeds the resulting JsonValue tree. Usage: %json~ {...} %%

• daslib/stringify.das — LongStringReader (visit): embeds a multi-line string literal without escaping. Usage: %stringify~ text %%

• daslib/spoof.das — SpoofTemplateReader (visit) + SpoofInstanceReader (suffix): a template engine that stores templates as strings and instantiates them by generating daslang source code at parse time.

See also

Full source: 11_reader_macro.das, reader_macro_mod.das

Previous tutorial: tutorial_macro_capture_macro

Next tutorial: tutorial_macro_typeinfo_macro

Standard library: daslib/regex_boost.das, daslib/json_boost.das, daslib/stringify.das, daslib/spoof.das

Language reference: Macros — full macro system documentation