8.19.27. Visual aids tour

Visual aids are the layer above ImGui that makes a running app self-narrating for tutorials, demos, and recordings. Five overlays draw on top of any ImGui frame:

  • Highlight — flash a colored rectangle around a widget’s bbox for N frames. Pinpoints “this thing right here.”

  • Mouse trail — fading line behind io.MousePos, so the synth cursor’s path through the UI is visible in a recording.

  • Cursor sprite — in-frame pointer drawn at io.MousePos. Without it, recordings show no cursor (the OS-level cursor doesn’t reach the framebuffer).

  • Narrate — sticky-note callout with an optional connector line to a target widget. Auto-fits to avoid sibling-widget overlap.

  • Auto-highlight on command — global flag that fires highlight on every accepted live command’s target. One-shot debug aid for figuring out which widget a curl invocation actually hit.

Plus two more for keyboard work — exercised by this tutorial’s recording:

  • Key HUD — bottom-center keycap pops for every synth key event, plus a Ctrl/Shift/Alt/Super modifier strip lit while a modifier is held.

  • Focus rect — colored rectangle around the widget that owns io.active_widget. Shows where typing lands.

This is the richer reference for the recording surface — the recording tutorial walked through the driver script anatomy; this one walks through what each visual aid does in isolation.

Source: modules/dasImgui/examples/tutorial/visual_aids_tour.das.

8.19.27.1. Walkthrough

The recording is voiced and self-verifying: each beat speaks a line while a real gesture fires under it and is asserted — clicking the highlight and narrate buttons (the click must register), scrubbing VOLUME (the value must change), and typing into NAME_INPUT then clearing it with Ctrl+A / Backspace (the buffer must end empty). A silently broken beat aborts the recording at teardown instead of shipping.

  1options gen2
  2options _comment_hygiene = true
  3options gc
  4
  5require imgui
  6require imgui_app
  7require opengl/opengl_boost
  8require live/glfw_live
  9require live/live_api
 10require live/live_commands
 11require live/live_vars
 12require live_host
 13require imgui/imgui_live
 14require imgui/imgui_boost_runtime
 15require imgui/imgui_boost_v2
 16require imgui/imgui_widgets_builtin
 17require imgui/imgui_containers_builtin
 18require imgui/imgui_visual_aids
 19
 20// =============================================================================
 21// TUTORIAL: visual_aids_tour — the keeper reference example for Phase 4/5.
 22//
 23// Main window holds four widgets being demonstrated:
 24//   - STATUS    : a text_show display (label-style)
 25//   - VOLUME    : a slider_float (interactive input)
 26//   - SAVE_BTN  : a button (trigger)
 27//   - NAME_INPUT: an input_text (typing target for the keyboard tour)
 28//
 29// Side "Controls" window has buttons that drive the visual aids:
 30//   - Highlight each demoed widget
 31//   - Toggle the mouse trail
 32//   - Pop a narrate callout pointing at VOLUME / SAVE_BTN
 33//   - Toggle auto-highlight-on-command (highlights every remote action)
 34//
 35// DRIVE (also exposed via curl on localhost:9090):
 36//   curl -X POST -d '{"name":"imgui_highlight","args":{"target":"VOLUME"}}' localhost:9090/command
 37//   curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' localhost:9090/command
 38//   curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SAVE_BTN","frames":180}}' localhost:9090/command
 39//   curl -X POST -d '{"name":"screenshot","args":{"file":"visual_aids_tour.png"}}' localhost:9090/command
 40//
 41// STANDALONE: daslang.exe modules/dasImgui/examples/tutorial/visual_aids_tour.das
 42// LIVE:       daslang-live modules/dasImgui/examples/tutorial/visual_aids_tour.das
 43// =============================================================================
 44
 45[export]
 46def init() {
 47    live_create_window("visual aids tour", 1000, 640)
 48    live_imgui_init(live_window)
 49    DisableIniPersistence()
 50    let io & = unsafe(GetIO())
 51    GetStyle().FontScaleMain = 1.5
 52}
 53
 54[export]
 55def update() { // nolint:STYLE038
 56    if (!live_begin_frame()) return
 57    begin_frame()
 58
 59    ImGui_ImplGlfw_NewFrame()
 60    apply_synth_io_override()
 61    NewFrame()
 62
 63    // ===== Subject window — the demoed widgets =====
 64    SetNextWindowPos(float2(40.0f, 40.0f), ImGuiCond.FirstUseEver)
 65    SetNextWindowSize(float2(440.0f, 360.0f), ImGuiCond.FirstUseEver)
 66    window(SUBJECT_WIN, (text = "Demoed widgets", closable = false,
 67                         flags = ImGuiWindowFlags.None)) {
 68        text("These widgets get highlighted, narrated, typed-into, etc.")
 69        separator(VA_SEP_1)
 70        text_show(STATUS, (value = "ready"))
 71        VOLUME.bounds = (0.0f, 100.0f)
 72        slider_float(VOLUME, (text = "Volume"))
 73        if (button(SAVE_BTN, (text = "Save"))) {
 74            print("save clicked, count={SAVE_BTN.click_count}\n")
 75        }
 76        separator(VA_SEP_2)
 77        input_text(NAME_INPUT, (text = "Name"))
 78        text("buffer = {NAME_INPUT.value}")
 79    }
 80
 81    // ===== Controls window — driver buttons for the aids =====
 82    SetNextWindowPos(float2(520.0f, 40.0f), ImGuiCond.FirstUseEver)
 83    SetNextWindowSize(float2(440.0f, 560.0f), ImGuiCond.FirstUseEver)
 84    window(CONTROLS_WIN, (text = "Visual aids controls", closable = false,
 85                          flags = ImGuiWindowFlags.None)) {
 86        text("Highlight a widget (yellow rect, 60 frames):")
 87        separator(VA_SEP_3)
 88        if (button(BTN_HIGHLIGHT_STATUS, (text = "highlight STATUS"))) {
 89            highlight("SUBJECT_WIN/STATUS")
 90        }
 91        if (button(BTN_HIGHLIGHT_VOLUME, (text = "highlight VOLUME"))) {
 92            highlight("SUBJECT_WIN/VOLUME")
 93        }
 94        if (button(BTN_HIGHLIGHT_SAVE, (text = "highlight SAVE_BTN"))) {
 95            highlight("SUBJECT_WIN/SAVE_BTN")
 96        }
 97        if (button(BTN_HIGHLIGHT_ALL, (text = "highlight all three"))) {
 98            highlight("SUBJECT_WIN/STATUS")
 99            highlight("SUBJECT_WIN/VOLUME")
100            highlight("SUBJECT_WIN/SAVE_BTN")
101        }
102
103        spacing(VA_SP_1)
104        text("Mouse trail (fading dots following cursor):")
105        separator(VA_SEP_4)
106        if (button(BTN_TRAIL_ON, (text = "mouse trail ON"))) {
107            mouse_trail(true)
108        }
109        if (button(BTN_TRAIL_OFF, (text = "mouse trail OFF"))) {
110            mouse_trail(false)
111        }
112
113        spacing(VA_SP_2)
114        text("Narrate callout (overlay box + connector line):")
115        separator(VA_SEP_5)
116        if (button(BTN_NARRATE_VOLUME, (text = "narrate VOLUME"))) {
117            narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME")
118        }
119        if (button(BTN_NARRATE_SAVE, (text = "narrate SAVE_BTN"))) {
120            narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")
121        }
122        if (button(BTN_NARRATE_FLOATING, (text = "narrate floating"))) {
123            narrate("Floating overlay, no target widget.")
124        }
125
126        spacing(VA_SP_3)
127        text("Auto-highlight on command (flag, currently {imgui_auto_highlight_on_command}):")
128        separator(VA_SEP_6)
129        if (button(BTN_AUTO_ON, (text = "auto-highlight ON"))) {
130            imgui_auto_highlight_on_command = true
131        }
132        if (button(BTN_AUTO_OFF, (text = "auto-highlight OFF"))) {
133            imgui_auto_highlight_on_command = false
134        }
135    }
136
137    end_of_frame()
138    Render()
139    var w, h : int
140    live_get_framebuffer_size(w, h)
141    glViewport(0, 0, w, h)
142    glClearColor(0.10f, 0.10f, 0.12f, 1.0f)
143    glClear(GL_COLOR_BUFFER_BIT)
144    live_imgui_render()
145
146    live_end_frame()
147}
148
149[export]
150def shutdown() {
151    live_imgui_shutdown()
152    live_destroy_window()
153}
154
155[export]
156def main() {
157    init()
158    while (!exit_requested()) {
159        update()
160        maybe_collect_gc()
161    }
162    shutdown()
163}

8.19.27.1.1. Layout

The subject has two window(...) containers:

  • SUBJECT_WIN (titled “Demoed widgets”) — the things the visual aids point at: STATUS (text_show), VOLUME (slider_float), SAVE_BTN (button), NAME_INPUT (input_text). Leaves register at SUBJECT_WIN/<ident>.

  • CONTROLS_WIN (titled “Visual aids controls”) — buttons that fire each aid in-process so you can iterate without a separate driver shell.

In the recording at the top, the driver drives the mouse aids through the controls window — it clicks BTN_HIGHLIGHT_* and BTN_NARRATE_* for real (each click verified), so the buttons’ own handlers fire highlight / narrate. The keyboard overlays (imgui_key_hud / imgui_focus_rect) and the typing/chords are posted as direct live commands. The buttons exist so a user dropped into daslang-live can drive every aid by hand for exploration too.

8.19.27.1.2. Highlight

A colored rectangle drawn around a widget’s bbox for N frames:

highlight("SUBJECT_WIN/VOLUME")              // default: yellow, 60 frames
highlight("SUBJECT_WIN/SAVE_BTN", 120, 0xFFFF8030u)   // orange, 120 frames

Highlights are short by design — long enough for “look here” to register, short enough that two consecutive highlights compose visually rather than queue. The recording clicks the three highlight buttons in turn (status, volume, save), each flashing a rect on the matching widget. Tunable defaults live in imgui_visual_aids.das (highlight_default_frames, highlight_color).

8.19.27.1.3. Mouse trail

A fading line behind io.MousePos, drawn every frame the trail is enabled:

mouse_trail(true)                // on, module defaults
mouse_trail(true, 0.45f)         // faster fade (alpha factor, default 0.96f)
mouse_trail(false)               // off — also clears the trail buffer

The trail’s value is mostly to make synth-cursor recordings parseable — without it, the cursor teleports between waypoints and the viewer can’t tell what happened. Real mouse motion shows a trail too, which is sometimes a nice touch in live demos.

8.19.27.1.4. Cursor sprite

A visible mouse-pointer sprite drawn at io.MousePos:

cursor_sprite(true)
cursor_sprite(false)

OS-level cursors don’t reach the framebuffer — the screen recorder sees ImGui’s draw output, not the cursor that the WM compositor draws on top. Without cursor_sprite, a recording shows widgets reacting to clicks with no visible “thing” doing the clicking. Always enable this before record_start for any tutorial recording.

8.19.27.1.5. Narrate

A sticky-note callout with optional connector line to a target widget:

narrate("Click here to save.", "SUBJECT_WIN/SAVE_BTN")   // 180 frames default
narrate("Drag this to set volume.", "SUBJECT_WIN/VOLUME", 240)
narrate("Floating overlay, no target.")                   // no connector line

The auto-fit logic tries four candidate anchors (right / left / below / above the target widget) and picks the first that doesn’t overlap the widget OR the viewport edge OR (when enabled) the key_hud zone. Falls back to right-then-clamp only when every candidate overflows. The result: a sticky-note that points at the right thing without covering it.

8.19.27.1.6. Auto-highlight on command

A module-scope flag that fires highlight on every accepted live command’s target:

imgui_auto_highlight_on_command = true
// any imgui_click / imgui_force_set / imgui_open ... now flashes its target

Useful when debugging “why didn’t my curl do anything” — turn it on, fire the command, and either see a highlight flash (command reached the right widget) or no flash (typo in the target path, or the widget isn’t in the registry that frame).

8.19.27.1.7. Key HUD + focus rect (recording’s keyboard tour)

Beyond the mouse-focused aids:

  • imgui_key_hud pops a keycap label at bottom-center for every synthesized key event. imgui_focus_rect draws a colored rectangle around whichever widget has keyboard focus right now.

  • The recording’s second half exercises both: a real click on NAME_INPUT lights the focus rect, then imgui_key_type types “Hello, World!” into the input — every keycap pops at the bottom with the matching mod-strip flash on H, W, and ! (auto-shift keys). The committed value is verified. Then imgui_key_chord fires Ctrl+A — Ctrl pill lights up while the “A” keycap pops — and a Backspace clears the (selected) buffer, verified empty.

  • All three keyboard live commands route through the L1 synth IO layer described in Driving from outside.

8.19.27.1.8. Standalone vs live

The visual-aid functions (highlight, mouse_trail, narrate, cursor_sprite) work in both modes — they’re just drawing code. The live-command wrappers (imgui_highlight / imgui_mouse_trail / etc.) need daslang-live for the HTTP surface.

8.19.27.1.9. Driving from outside

Every aid is reachable via curl:

curl -X POST -d '{"name":"imgui_highlight","args":{"target":"SUBJECT_WIN/VOLUME"}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_mouse_trail","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_cursor_sprite","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_narrate","args":{"text":"click me","target":"SUBJECT_WIN/SAVE_BTN","frames":180}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_auto_highlight","args":{"enabled":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_key_hud","args":{"enabled":true,"show_modifiers":true}}' \
     localhost:9090/command
curl -X POST -d '{"name":"imgui_focus_rect","args":{"enabled":true}}' \
     localhost:9090/command

The recording at the top of this page fires this exact sequence (plus the keyboard tour) through the playwright transport instead of curl.

See also

Full source: modules/dasImgui/examples/tutorial/visual_aids_tour.das

Driver: modules/dasImgui/tests/record_visual_aids_tour.das — the voiced, self-verifying tour that walks every aid in turn.

Implementation: modules/dasImgui/widgets/imgui_visual_aids.das — the full surface plus narrate auto-fit, key HUD, focus rect.

Previous tutorial: Driving from outside

Next tutorial: Recording tutorial videos

Boost macros — the macro layer.