8.7.11. dasLLAMA-10 — Thinking Models

A thinking model works through the question before answering, and the reply’s wire format carries that reasoning span: Qwen3 and GLM wrap it in <think>...</think>, gpt-oss speaks Harmony channels, gemma-4 a thought channel. This tutorial splits replies into reasoning and answer — buffered and live-streamed — and then turns thinking off.

Run it with a thinking-family GGUF (Qwen3-0.6B-Q8_0 works):

daslang.exe -jit tutorials/dasLLAMA/10_thinking.das -- path/to/Qwen3-0.6B-Q8_0.gguf

8.7.11.1. The reply carries its reasoning

respond streams the raw pieces — reasoning markers included — while chat.history stores the reply already reasoning-stripped (tutorial 02). Thinking burns token budget, so raise max_new past its 256 default:

var chat = create_chat(m, "You are a concise assistant.", 1024l)
add_user(chat, "Which is larger: 17 * 24 or 20 * 20? One sentence.")
let full = respond(m, chat, SamplingParams()) $(_piece) => true   // buffered: collect only
// full starts "<think> ..." — the raw wire; history stores only the answer

8.7.11.2. split_reasoning: the buffered split

split_reasoning splits a complete reply at the family’s reasoning boundary. Both halves come back stripped when a reasoning span was found; a reply with no reasoning passes through untouched — so it is safe to run on every reply, thinking model or not.

let sp = split_reasoning(chat, full)
print("reasoning: {sp.reasoning}\n")   // the model's work: 17*24 = 408, 20*20 = 400 ...
print("content:   {sp.content}\n")     // the answer: 17 * 24 is larger than 20 * 20.

8.7.11.3. Streaming the split: think_feed / think_finish

A live UI wants the reasoning marked while it streams, not after. The ThinkStream splitter turns each streamed piece into (reasoning, content) deltas, holding partial markers across piece boundaries. Create it for the turn, feed every piece, and flush at the end — an unclosed reasoning span (the token budget cut mid-thought) classifies as reasoning:

var chat2 = create_chat(m, "You are a concise assistant.", 1024l)
add_user(chat2, "What is the capital of Australia? One sentence.")
var ts = make_think_stream(chat2)
respond(m, chat2, SamplingParams()) $(piece) {
    var r = ""
    var c = ""
    think_feed(ts, piece, r, c)   // overwritten per piece; either may be empty
    fprint(fstdout(), c)          // show the answer live; dim or hide r as you like
    fflush(fstdout())
    return true
}
var tail_r = ""
var tail_c = ""
think_finish(ts, tail_r, tail_c)

On a model with no reasoning format the stream is a pass-through: every piece comes back as content. And a small model sometimes ends a later turn inside its think block — the flush then returns the tail as reasoning, and the content half stays empty; the tutorial file shows the pattern that handles both endings.

8.7.11.4. Turning thinking off

Hybrid families answer directly when the turn opens with the template’s empty think block. set_thinking(false) renders exactly that (a no-op for models with no think specials in the vocabulary); the default is on. Schedulers that cut streams themselves read effective_stop_ids — the template’s stops plus the thinking-off extras while thinking is off — rather than the template’s raw stop list:

set_thinking(chat, false)
let stops <- effective_stop_ids(chat)
add_user(chat, "And 12 * 12 vs 11 * 13? One sentence.")
// respond(...) now answers directly — no <think> span on the wire