8.18.1.27. Visual aids tour

Visual aids are the layer above ImGui that makes a running app self-narrating for tutorials, demos, and recordings. Five overlays draw on top of any ImGui frame:

  • Highlight — flash a colored rectangle around a widget’s bbox for N frames. Pinpoints “this thing right here.”

  • Mouse trail — fading line behind io.MousePos, so the synth cursor’s path through the UI is visible in a recording.

  • Cursor sprite — in-frame pointer drawn at io.MousePos. Without it, recordings show no cursor (the OS-level cursor doesn’t reach the framebuffer).

  • Narrate — sticky-note callout with an optional connector line to a target widget. Auto-fits to avoid sibling-widget overlap.

  • Auto-highlight on command — global flag that fires highlight on every accepted live command’s target. One-shot debug aid for figuring out which widget a curl invocation actually hit.

Plus two more for keyboard work — exercised by this tutorial’s recording:

  • Key HUD — bottom-center keycap pops for every synth key event, plus a Ctrl/Shift/Alt/Super modifier strip lit while a modifier is held.

  • Focus rect — colored rectangle around the widget that owns io.active_widget. Shows where typing lands.

This is the richer reference for the recording surface — the recording tutorial walked through the driver script anatomy; this one walks through what each visual aid does in isolation.

Source: modules/dasImgui/examples/tutorial/visual_aids_tour.das.

8.18.1.27.1. Walkthrough

The recording is voiced and self-verifying: each beat speaks a line while a real gesture fires under it and is asserted — clicking the highlight and narrate buttons (the click must register), scrubbing VOLUME (the value must change), and typing into NAME_INPUT then clearing it with Ctrl+A / Backspace (the buffer must end empty). A silently broken beat aborts the recording at teardown instead of shipping.

  1options gen2
  2options _comment_hygiene = true
  3
  4require imgui
  5require imgui_app
  6require opengl/opengl_boost
  7require live/glfw_live
  8require live/live_api
  9require live/live_commands
 10require live/live_vars
 11require live_host
 12require imgui/imgui_live
 13require imgui/imgui_boost_runtime
 14require imgui/imgui_boost_v2
 15require imgui/imgui_widgets_builtin
 16require imgui/imgui_containers_builtin
 17require imgui/imgui_visual_aids
 18
 19// =============================================================================
 20// TUTORIAL: visual_aids_tour — the keeper reference example for Phase 4/5.
 21//
 22// Main window holds four widgets being demonstrated:
 23//   - STATUS    : a text_show display (label-style)
 24//   - VOLUME    : a slider_float (interactive input)
 25//   - SAVE_BTN  : a button (trigger)
 26//   - NAME_INPUT: an input_text (typing target for the keyboard tour)
 27//
 28// Side "Controls" window has buttons that drive the visual aids:
 29//   - Highlight each demoed widget
 30//   - Toggle the mouse trail
 31//   - Pop a narrate callout pointing at VOLUME / SAVE_BTN
 32//   - Toggle auto-highlight-on-command (highlights every remote action)
 33//
 34// DRIVE (also exposed via curl on localhost:9090):
 35//   curl -X POST -d '{"name":"imgui_highlight","args":{"target":"VOLUME"}}' localhost:9090/command
 36//   curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' localhost:9090/command
 37//   curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SAVE_BTN","frames":180}}' localhost:9090/command
 38//   curl -X POST -d '{"name":"screenshot","args":{"file":"visual_aids_tour.png"}}' localhost:9090/command
 39//
 40// STANDALONE: daslang.exe modules/dasImgui/examples/tutorial/visual_aids_tour.das
 41// LIVE:       daslang-live modules/dasImgui/examples/tutorial/visual_aids_tour.das
 42// =============================================================================
 43
 44[export]
 45def init() {
 46    live_create_window("visual aids tour", 1000, 640)
 47    live_imgui_init(live_window)
 48    DisableIniPersistence()
 49    let io & = unsafe(GetIO())
 50    GetStyle().FontScaleMain = 1.5
 51}
 52
 53[export]
 54def update() { // nolint:STYLE038
 55    if (!live_begin_frame()) return
 56    begin_frame()
 57
 58    ImGui_ImplGlfw_NewFrame()
 59    apply_synth_io_override()
 60    NewFrame()
 61
 62    // ===== Subject window — the demoed widgets =====
 63    SetNextWindowPos(float2(40.0f, 40.0f), ImGuiCond.FirstUseEver)
 64    SetNextWindowSize(float2(440.0f, 360.0f), ImGuiCond.FirstUseEver)
 65    window(SUBJECT_WIN, (text = "Demoed widgets", closable = false,
 66                         flags = ImGuiWindowFlags.None)) {
 67        text("These widgets get highlighted, narrated, typed-into, etc.")
 68        separator(VA_SEP_1)
 69        text_show(STATUS, (value = "ready"))
 70        VOLUME.bounds = (0.0f, 100.0f)
 71        slider_float(VOLUME, (text = "Volume"))
 72        if (button(SAVE_BTN, (text = "Save"))) {
 73            print("save clicked, count={SAVE_BTN.click_count}\n")
 74        }
 75        separator(VA_SEP_2)
 76        input_text(NAME_INPUT, (text = "Name"))
 77        text("buffer = {NAME_INPUT.value}")
 78    }
 79
 80    // ===== Controls window — driver buttons for the aids =====
 81    SetNextWindowPos(float2(520.0f, 40.0f), ImGuiCond.FirstUseEver)
 82    SetNextWindowSize(float2(440.0f, 560.0f), ImGuiCond.FirstUseEver)
 83    window(CONTROLS_WIN, (text = "Visual aids controls", closable = false,
 84                          flags = ImGuiWindowFlags.None)) {
 85        text("Highlight a widget (yellow rect, 60 frames):")
 86        separator(VA_SEP_3)
 87        if (button(BTN_HIGHLIGHT_STATUS, (text = "highlight STATUS"))) {
 88            highlight("SUBJECT_WIN/STATUS")
 89        }
 90        if (button(BTN_HIGHLIGHT_VOLUME, (text = "highlight VOLUME"))) {
 91            highlight("SUBJECT_WIN/VOLUME")
 92        }
 93        if (button(BTN_HIGHLIGHT_SAVE, (text = "highlight SAVE_BTN"))) {
 94            highlight("SUBJECT_WIN/SAVE_BTN")
 95        }
 96        if (button(BTN_HIGHLIGHT_ALL, (text = "highlight all three"))) {
 97            highlight("SUBJECT_WIN/STATUS")
 98            highlight("SUBJECT_WIN/VOLUME")
 99            highlight("SUBJECT_WIN/SAVE_BTN")
100        }
101
102        spacing(VA_SP_1)
103        text("Mouse trail (fading dots following cursor):")
104        separator(VA_SEP_4)
105        if (button(BTN_TRAIL_ON, (text = "mouse trail ON"))) {
106            mouse_trail(true)
107        }
108        if (button(BTN_TRAIL_OFF, (text = "mouse trail OFF"))) {
109            mouse_trail(false)
110        }
111
112        spacing(VA_SP_2)
113        text("Narrate callout (overlay box + connector line):")
114        separator(VA_SEP_5)
115        if (button(BTN_NARRATE_VOLUME, (text = "narrate VOLUME"))) {
116            narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME")
117        }
118        if (button(BTN_NARRATE_SAVE, (text = "narrate SAVE_BTN"))) {
119            narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")
120        }
121        if (button(BTN_NARRATE_FLOATING, (text = "narrate floating"))) {
122            narrate("Floating overlay, no target widget.")
123        }
124
125        spacing(VA_SP_3)
126        text("Auto-highlight on command (flag, currently {imgui_auto_highlight_on_command}):")
127        separator(VA_SEP_6)
128        if (button(BTN_AUTO_ON, (text = "auto-highlight ON"))) {
129            imgui_auto_highlight_on_command = true
130        }
131        if (button(BTN_AUTO_OFF, (text = "auto-highlight OFF"))) {
132            imgui_auto_highlight_on_command = false
133        }
134    }
135
136    end_of_frame()
137    Render()
138    var w, h : int
139    live_get_framebuffer_size(w, h)
140    glViewport(0, 0, w, h)
141    glClearColor(0.10f, 0.10f, 0.12f, 1.0f)
142    glClear(GL_COLOR_BUFFER_BIT)
143    live_imgui_render()
144
145    live_end_frame()
146}
147
148[export]
149def shutdown() {
150    live_imgui_shutdown()
151    live_destroy_window()
152}
153
154[export]
155def main() {
156    init()
157    while (!exit_requested()) {
158        update()
159    }
160    shutdown()
161}

8.18.1.27.1.1. Layout

The subject has two window(...) containers:

  • SUBJECT_WIN (titled “Demoed widgets”) — the things the visual aids point at: STATUS (text_show), VOLUME (slider_float), SAVE_BTN (button), NAME_INPUT (input_text). Leaves register at SUBJECT_WIN/<ident>.

  • CONTROLS_WIN (titled “Visual aids controls”) — buttons that fire each aid in-process so you can iterate without a separate driver shell.

In the recording at the top, the driver drives the mouse aids through the controls window — it clicks BTN_HIGHLIGHT_* and BTN_NARRATE_* for real (each click verified), so the buttons’ own handlers fire highlight / narrate. The keyboard overlays (imgui_key_hud / imgui_focus_rect) and the typing/chords are posted as direct live commands. The buttons exist so a user dropped into daslang-live can drive every aid by hand for exploration too.

8.18.1.27.1.2. Highlight

A colored rectangle drawn around a widget’s bbox for N frames:

highlight("SUBJECT_WIN/VOLUME")              // default: yellow, 60 frames
highlight("SUBJECT_WIN/SAVE_BTN", 120, 0xFFFF8030u)   // orange, 120 frames

Highlights are short by design — long enough for “look here” to register, short enough that two consecutive highlights compose visually rather than queue. The recording clicks the three highlight buttons in turn (status, volume, save), each flashing a rect on the matching widget. Tunable defaults live in imgui_visual_aids.das (highlight_default_frames, highlight_color).

8.18.1.27.1.3. Mouse trail

A fading line behind io.MousePos, drawn every frame the trail is enabled:

mouse_trail(true)                // on
mouse_trail(true, 0.45f)         // 450ms fade
mouse_trail(false)               // off

The trail’s value is mostly to make synth-cursor recordings parseable — without it, the cursor teleports between waypoints and the viewer can’t tell what happened. Real mouse motion shows a trail too, which is sometimes a nice touch in live demos.

8.18.1.27.1.4. Cursor sprite

A visible mouse-pointer sprite drawn at io.MousePos:

cursor_sprite(true)
cursor_sprite(false)

OS-level cursors don’t reach the framebuffer — the screen recorder sees ImGui’s draw output, not the cursor that the WM compositor draws on top. Without cursor_sprite, a recording shows widgets reacting to clicks with no visible “thing” doing the clicking. Always enable this before record_start for any tutorial recording.

8.18.1.27.1.5. Narrate

A sticky-note callout with optional connector line to a target widget:

narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")   // 180 frames default
narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME", 240)
narrate("Floating overlay, no target.")                   // no connector line

The auto-fit logic tries four candidate anchors (right / left / below / above the target widget) and picks the first that doesn’t overlap the widget OR the viewport edge OR (when enabled) the key_hud zone. Falls back to right-then-clamp only when every candidate overflows. The result: a sticky-note that points at the right thing without covering it.

8.18.1.27.1.6. Auto-highlight on command

A module-scope flag that fires highlight on every accepted live command’s target:

imgui_auto_highlight_on_command = true
// any imgui_click / imgui_force_set / imgui_open ... now flashes its target

Useful when debugging “why didn’t my curl do anything” — turn it on, fire the command, and either see a highlight flash (command reached the right widget) or no flash (typo in the target path, or the widget isn’t in the registry that frame).

8.18.1.27.1.7. Key HUD + focus rect (recording’s keyboard tour)

Beyond the mouse-focused aids:

  • imgui_key_hud pops a keycap label at bottom-center for every synthesized key event. imgui_focus_rect draws a colored rectangle around whichever widget has keyboard focus right now.

  • The recording’s second half exercises both: a real click on NAME_INPUT lights the focus rect, then imgui_key_type types “Hello, World!” into the input — every keycap pops at the bottom with the matching mod-strip flash on H, W, and ! (auto-shift keys). The committed value is verified. Then imgui_key_chord fires Ctrl+A — Ctrl pill lights up while the “A” keycap pops — and a Backspace clears the (selected) buffer, verified empty.

  • All three keyboard live commands route through the L1 synth IO layer described in Driving from outside.

8.18.1.27.1.8. Standalone vs live

The visual-aid functions (highlight, mouse_trail, narrate, cursor_sprite) work in both modes — they’re just drawing code. The live-command wrappers (imgui_highlight / imgui_mouse_trail / etc.) need daslang-live for the HTTP surface.

8.18.1.27.1.9. Driving from outside

Every aid is reachable via curl:

curl -X POST -d '{"name":"imgui_highlight","args":{"target":"SUBJECT_WIN/VOLUME"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_cursor_sprite","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SUBJECT_WIN/SAVE_BTN","frames":180}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_auto_highlight","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_key_hud","args":{"enabled":true,"show_modifiers":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_focus_rect","args":{"enabled":true}}' \
     localhost:9090/command

The recording at the top of this page fires this exact sequence (plus the keyboard tour) through the playwright transport instead of curl.

See also

Full source: modules/dasImgui/examples/tutorial/visual_aids_tour.das

Driver: modules/dasImgui/tests/record_visual_aids_tour.das — the voiced, self-verifying tour that walks every aid in turn.

Implementation: modules/dasImgui/widgets/imgui_visual_aids.das — the full surface plus narrate auto-fit, key HUD, focus rect.

Previous tutorial: Driving from outside

Next tutorial: Recording tutorial videos

Boost macros — the macro layer.