7.20. dasllama-server — an OpenAI-compatible server over dasLLAMA

dasllama-server (directory: utils/dasllama-server/) is a drop-in OpenAI-compatible HTTP server for dasLLAMA inference - on the CPU, or with the Metal and Vulkan GPU tiers - written entirely in daslang over the public dasllama facade plus the dasHV HTTP layer. Point any OpenAI client (opencode, Open WebUI, the llm CLI, the openai Python SDK, …) at http://127.0.0.1:<port>/v1.

It reaches only public facade verbs — load_model, create_chat_renderer / add_user / render_assistant / render_turn, eval_batch through dasllama_scheduler, transcribe, embed — and that is the point: the server is the acceptance test for the dasLLAMA API. If a full OpenAI surface builds with no reach into engine internals, the facade is complete.

7.20.1. Get it

The server ships as the dasllama package, refreshed with every daslang release:

Platform	Install
macOS, Apple silicon	brew install --cask borisbat/daslang/dasllama (the tray app), or brew install borisbat/daslang/dasllama (the commands)
Windows x64	scoop bucket add daslang https://github.com/borisbat/scoop-daslang, then scoop install dasllama
Debian, Ubuntu	sudo apt install ./dasllama_<version>_<arch>.deb, from the release page
Fedora	sudo dnf install ./dasllama-<version>-1.<arch>.rpm, from the release page
Linux (Homebrew)	brew install borisbat/daslang/dasllama
any, with Python	pip install dasllama

Or unpack an archive from the rolling dasllama-server release, https://github.com/GaijinEntertainment/daScript/releases/tag/dasllama-server: dasllama-darwin-arm64.zip (macOS, Apple silicon), dasllama-windows-x64.zip, dasllama-linux-x86_64.tar.gz and dasllama-linux-arm64.tar.gz.

Every form carries the same four programs. A package manager puts them on PATH as dasllama-server, dasllama-cli, dasllama-bench and dasllama-watchdog. Inside an archive they are dasllama-server.exe, dasllama-cli.exe, dasllama-bench.exe and watchdog (watchdog.exe on Windows) on Linux and Windows alike; the macOS .app carries them without the suffix in Contents/MacOS.

The Linux builds need glibc 2.35 or newer (Debian 12, Ubuntu 22.04 and later), the system OpenSSL 3 (libssl3 on Debian and Ubuntu, openssl-libs on Fedora) and curl for the catalog downloads - the .deb and .rpm pull them in. The macOS and Windows builds carry their own OpenSSL.

Start the supervisor - dasllama-watchdog (watchdog in an archive), or on macOS the dasllama-server.app itself, whose launcher is the watchdog. brew services start dasllama starts it at login. The watchdog keeps the server up, puts the dasllama mark in the notification area, and a click on it opens the control page at http://127.0.0.1:8080/. With no model configured the server starts in setup mode: pick a model from the catalog, it downloads into ~/.dasllama/models, and serve this model restarts into it.

Everything the programs write lives under ~/.dasllama (%USERPROFILE%\.dasllama on Windows), never beside them. So an installed copy stays read-only, and an upgrade keeps it all:

Path	What
models/	the catalog downloads, and the .dlim images baked beside them
dasllama-server.toml	the config the control page saves
tune/<program>.tune.json	the box’s runtime section: the Metal kernel choices raced at the first start, the knobs
logs/	dasllama-server.log, dasllama-cli.log, the watchdog’s log and crash bundles, libhv’s daily log

--config names a config outright. Without it, the programs look for dasllama-server.toml in the working directory, then in ~/.dasllama, then beside the program. A config or sidecar an earlier download kept beside the exe still reads - the sidecar is copied to tune/ at the first start. A JIT run from the SDK keeps its logs in the SDK’s logs/.

The exe is a fat build (daspkg release --fat): plain code for the platform’s baseline CPU class - x86-avx2 on x86, arm-neon on arm64 - with one clone of every [tune] kernel per class the engine ships a profile for (x86-vnni512, x86-amx, arm-i8mm), picked from cpuid at start. On a Mac the Metal kernel choices are raced once at the first start and kept in ~/.dasllama/tune/. That is the good default, and it is as far as a solid executable goes: a fat build cannot tune - its kernels are baked, it carries no tuner, and --tune, the sidecar exchange and a re-tune have nothing to act on (the control page’s exchange levers refuse). To tune every kernel for this box, install daslang with dasLLAMA (the SDK) and run the server through the JIT - the command under Run from the source tree - or daspkg release the package on that box: either mints the box’s own sidecar, every kernel raced on your own hardware instead of picked from a class profile.

The builds are not code-signed. macOS asks once under System Settings > Privacy & Security before the cask’s first start. An app unpacked from a downloaded archive is quarantined, and started from the download folder it runs from a read-only copy - so first drag dasllama-server.app out of Downloads with the Finder, then either allow it under System Settings > Privacy & Security after the first refused start, or clear the flag once with xattr -dr com.apple.quarantine dasllama-server.app. Windows SmartScreen shows an unknown publisher: More info, Run anyway.

7.20.2. Measure your box

The control page’s benchmark button runs the llama-bench rows on the served model right in the server - pp512 and tg128, three timed reps each after an untimed warmup, and on a slot that serves two streams or more tg128@N, N streams (up to four) through the scheduler, the summed rate - and shows tokens per second beside the hardware line. Every route that could contend holds for the run (the text, audio and speech routes answer 503, the model-switching, bake and download routes 409). A run is seconds on a small model. The same rows from a shell, and the llama.cpp comparison the page cannot run itself, come from dasllama-bench (on PATH from a package; in an archive dasllama-bench.exe beside the server on Linux and Windows, dasllama-server.app/Contents/MacOS/dasllama-bench on a Mac):

dasllama-bench -m ~/.dasllama/models/<model>.gguf           # pp512/tg128, five reps
dasllama-bench -m <model.gguf> --ref /path/to/llama-bench   # and llama.cpp on the same GGUF

The two run the same rows, not the same numbers: the page measures the served slot as it is - its device, its KV codec - while dasllama-bench runs f16 KV on the CPU unless -ngl says otherwise, five reps to the page’s three. The page’s result names what produced it: the device that served the rows, the KV codec, the exec tier (exe-native in a bundle, jit from the SDK), the tune state (fat in a bundle; tuned or untuned under the JIT), and the llama-bench line that matches it.

7.20.3. Run from the source tree

Run under -jit — the interpreter is refused, it is far too slow for inference:

bin/daslang -jit utils/dasllama-server/main.das -- --model <model.gguf> [--port 8080] [--quant q8] \
    [--asr <asr.bin>] [--asr-workers 2] [--mmproj <mmproj.gguf>] [--image-mmproj <mmproj.gguf>] \
    [--audio-mmproj <mmproj.gguf>] \
    [--ctx 4096] [--streams 4] [--chunk 512] [--chunk-idle 2048] [--page-rows 64] [--prefix N] \
    [--prefix-states 256] [--prefix-state-mb 0] [--tune]

Flag	Short	Default	Meaning
--config	-c	auto	TOML config file; keys mirror the long flag names, explicit CLI flags override. Without the flag, a dasllama-server.toml in the cwd, in ~/.dasllama or next to the program loads automatically, the first found
--model	-m	(required)	GGUF model to serve (here or in --config)
--port	-p	8080	Listen port
--host		127.0.0.1	Interface to listen on. The default serves this machine only; 0.0.0.0 opens the server to every device on the network, which is the operator’s choice, never the default
--quant	-q	q8	Weight serving mode: fp32 | q8 | q4 (the legacy requant tier); the file-format spellings q8_0 | q4_0 | q4_k | q5_k | q6_k | mxfp4 | f16 | bf16 select q8, where those planes serve natively
--kv-dtype		f16	KV-cache codec: f32 | f16 | q8_0 | tq4 (rotated 4-bit; needs a power-of-two head size). Under --gpu vulkan the whole-model driver holds its cache in the same codec for f16, q8_0 and tq4: q8_0 is about half the f16 cache’s VRAM and tq4 about a quarter, which a longer --ctx or more resident layers take
--gpu		auto	GPU backend: auto (metal or vulkan when detected, else CPU) | off | metal | metal-required | vulkan. vulkan serves a model that fits the card whole from the GPU, every stream’s cache with it; the control page’s model card says what a slot got. Details under The gpu key
--metal		off	Metal serving mode when --gpu is unset: off | auto (the GPU when a path’s gates pass, the CPU otherwise) | required (a call the GPU cannot serve is a hard failure). A set flag keeps the legacy env-driven path
--gpu-layers		auto	vulkan: resident MoE expert-stack layers, offloaded from the end (config key gpu_layers; The gpu key)
--gpu-stream		auto	vulkan: streamed prefill MoE layers below the resident set (config key gpu_stream)
--gpu-dn		on	vulkan: deltanet (recurrent) layers through the device chain
--gpu-attn		on	vulkan: full-attention layers through the device chain
--gpu-dense		off	vulkan: dense attention-side planes resident
--gpu-vram-mb		device	vulkan: resident-weight VRAM cap override in MB (default: query the device)
--asr	-a	—	ASR model (whisper / parakeet / qwen3-asr) — enables the /v1/audio/* routes. Repeatable: each model serves under its file’s stem, the first is the default (--mmproj and --asr-workers are the first one’s); the config’s [[asr]] tables give each model its own id, mmproj and workers
--asr-workers		1	Long-lived ASR request threads; each owns a model and a reusable session. 2 permits two parallel transcriptions. With several ASR models, the first one’s
--mmproj		—	mmproj GGUF for the Qwen3-ASR route (paired with --asr; with several ASR models, the first one’s)
--tts		—	TTS model GGUF (kitten-nano, kitten-mini, kokoro-82m with the front-end packs tts_g2p.bin + tts_postag.bin beside them; or a pocket-tts-<lang>-q8 file, which reads text, needs no pack and clones a voice) — enables /v1/audio/speech. Repeatable: each model serves under its file’s stem, the first is the default (--tts-lane and --tts-voices-dir are the first one’s); the config’s [[tts]] tables give each model its own id, lane and voices_dir
--tts-lane		q8	Weight lane the speech worker pins around its load: q8 (the prepared quant image beside the GGUF) or f32 (the file’s own planes). An unknown spelling warns and serves q8. With several TTS models, the first one’s
--tts-voices-dir		—	Directory of voice clips (wav / flac / mp3 / ogg, a few seconds of one speaker each, 60 s at most) a cloning speech model adds to its voices at boot (with several TTS models, the first one’s), each under its file’s stem; a clip that does not decode or runs past 60 s is logged and skipped, the key on a model that cannot clone is logged and ignored
--image-mmproj		—	Vision mmproj (gemma4uv, gemma4v or gemma3v, sniffed) for the default model — the chat route then accepts image_url content parts. When the file also carries an audio encoder (the gemma-4 E-series and the Qwen Omni mmprojs carry both towers) the same flag arms native audio: input_audio parts serve through the same slot
--audio-mmproj		—	Audio mmproj for the default model — the chat route then accepts input_audio content parts: a whisper-class tower with its projector (Qwen2-Audio, Qwen2.5-Omni, Ultravox, Voxtral) or the gemma-4 Conformer. Not needed when --image-mmproj names a file that carries the audio tower too
--ctx		model	Context length in tokens, served whole: the whole-model GPU driver holds it for every stream or declines to the per-op rails, and the load log names the room. Default: the model’s trained context_length, shortened to what the card holds (the log says by how much and why). A Metal model served from its image alone (no host weights) is cut to the rows its device KV mirror holds (DASLLAMA_METAL_KV_MIRROR_MB), with a warning — a boot load and a live /v1/models/load alike.
--max-tokens		16384	Default reply token budget when a request omits max_tokens (clamped to --ctx per request)
--streams	-s	4	Max concurrent generation streams
--threads	-t	16	Worker-lane cap for the matmul dispatch (-1 = all cores). Decode is bandwidth-bound, so an uncapped dispatch only fights the rest of the box
--team-dispatch		hybrid	hybrid: the LLM uses the worker team; the ASR workers use it too when every LLM slot decodes on a GPU (Metal, or the whole model resident on a Vulkan device) and run inline otherwise; the TTS worker runs inline; team: every caller uses serialized team publishes; inline: every caller runs independently
--affinity		-1	Worker CPU affinity: -1 = the platform default (QoS on darwin, off elsewhere), 0 = off, 1 = an ideal-CPU hint, 2 = a hard mask (DAS_JOBQUE_AFFINITY overrides)
--chunk		backend	Prefill quantum in tokens while a stream is decoding: decode stalls at most this many per tick. Default: 512 where a GPU prefills (a window’s cost is mostly fixed there, so a small chunk runs a prompt at a fraction of the window’s rate), 64 on the CPU (a chunk costs its token count, so a short one keeps the stall short at 13 - 29% of the prompt’s rate)
--chunk-idle		backend	Prefill quantum in tokens while no stream is decoding (never under --chunk): nothing waits on the tick, so the window can be wide. Default: 2048 on Metal, 512 on the other backends
--page-rows		64	KV page size in positions for paged serving
--prefix		auto	Prefix-cache retention cap in pages (auto: one full context per stream; -1 = unbounded)
--prefix-states		256	Recurrent models (the Qwen3.5/3.6/3.8 hybrids) cache whole checkpoints instead of pages: the most a slot keeps (0 = none; config key prefix_states); the byte budget binds first. Every request leaves one at its opening’s shared head (the system text or the tool block, whichever its template writes first), one at its opening with the tools, one where its last message starts, one where it ends (ahead of the generation prompt), one at its finished turn; a conversation’s own stops are dropped before shared openings
--prefix-state-mb		auto	The checkpoints’ byte budget in MB, snapshots and held pages (auto: a quarter of the box’s RAM - in a container, of its memory limit where that is lower; config key prefix_state_mb)
--flat		—	Flat preallocated KV sessions — disables paged serving and the prefix cache
--mtp		off	MTP/NextN self-speculative decode. Unset, a slot leaves it off, except one that runs one stream (streams = 1) on a GPU the model’s catalog row lists under serve_mtp (modules/dasLLAMA/performance/model_specs.das: the platforms, metal or vulkan, where that model’s self-speculation measured a win) — Metal, or the whole model resident on a Vulkan device with its routed experts on the card. A model with no catalog row, a model whose row lists another platform, several streams, the CPU, and a MoE whose routed experts the resident driver sums on the host all decode plain: on the CPU the round’s second verify row costs a second decode step, at several streams the plain batched step is faster, and on a MoE at one stream the draft overhead can cost more than the accepted rows buy. A drafting Vulkan slot (--gpu vulkan) keeps its streams’ caches on the card (device-home) where the whole-model driver runs the draft head, its verify and a rejected round’s rollback on the device; where any of them stays on the host (the load log names which), the slot serves host-cached sessions. true / false set it outright. It needs a model with an in-file NextN head (the -MTP- GGUFs), or an assistant drafter GGUF beside the model file (gemma-4: mtp-<stem>-Q8_0.gguf), which the load attaches; on any other model the server logs one line and serves plain. Greedy requests are output-invariant; a sampled request draws each verify row with its own sampler and keeps the plain sampled distribution, at a lower acceptance rate. /v1/stats reports mtp_drafted / mtp_accepted. A constrained reply (response_format, a forced tool_choice) decodes plain.
--rope-scaling		file	RoPE scaling override for the load: yarn | linear | none; unset keeps the model file’s own rope.scaling keys, none drops them (a file’s per-pair factor tensors stay, as llama.cpp keeps them). yarn folds the NTK-by-parts frequency ramp and the 1 + 0.1 ln(s) magnitude into the rope tables the way llama.cpp’s --rope-scaling yarn does. The Qwen families publish the recipe (Qwen2.5-Instruct 7B and up, Qwen3, Qwen3-Next / 3.5 / 3.8: factor 4 over the trained context) and ship no scaling keys because static YaRN costs a little on short texts - arm it when a conversation needs the length; no other vendor validates it, and a non-Qwen file logs a warning. The override is baked into the prepared image under its own lane (model.gguf.metal-yarn4.dlim), so the first load with it mints once. Per-model in a [[models]] roster: rope_scaling (config key rope_scaling)
--rope-scale		file	The scaling factor s (the context multiplier) for the override; unset reads the file’s rope.scaling.factor, and yarn needs one (config key rope_scale)
--yarn-orig-ctx		file	YaRN: the original training context the factor extends; unset reads the file’s original_context_length, else its context_length (config key yarn_orig_ctx; llama.cpp’s --yarn-orig-ctx)
--lcpp-bin		—	Path to a llama.cpp llama-bench binary: the control page’s benchmark button then runs the A/B child (our bench, then llama-bench on the same GGUF) instead of the in-process rows; source-tree daslang only (config key lcpp_bin)
--models-dir		~/.dasllama/models	Where the model catalog downloads land (DASLLAMA_MODELS_DIR overrides both this and the config key)
--tune		—	Re-tune this box’s dasLLAMA kernels, then relaunch (see Per-box tuning; a fat build carries no tuner and ignores it)
--help	-?	—	Show help and exit

A config file replaces long command lines; keys are the long flag names with underscores. A dasllama-server.toml in the cwd, in ~/.dasllama, or next to the program loads automatically, the first one found; --config other.toml picks a different one:

model = "D:/models/SmolLM2-135M-Instruct-Q8_0.gguf"
port = 8080
host = "127.0.0.1"   # the interface: this machine only; "0.0.0.0" opens it to the network
quant = "q8"
kv_dtype = "tq4"   # rotated 4-bit KV - half the q8_0 cache bytes
ctx = 4096
max_tokens = 4096  # default reply budget for clients that omit max_tokens
streams = 4
threads = 16       # matmul dispatch lane cap; -1 = all cores
team_dispatch = "hybrid"
asr_workers = 2    # two independent transcription requests; each worker owns an ASR model

Several speech models serve side by side, each on worker threads of its own, so a request to one never waits behind another. asr and tts are then rosters instead of paths — a table a model, the first the default (the model a request with no model field gets):

[[asr]]
path = "models/parakeet-tdt-0.6b-v3.bin"

[[asr]]
id = "qwen-asr"                              # the id requests name; default: the file's stem
path = "models/Qwen3-ASR-1.7B-Q8_0.gguf"
mmproj = "models/mmproj-Qwen3-ASR-1.7B-bf16.gguf"
workers = 2

[[tts]]
path = "models/kokoro-82m.gguf"

[[tts]]
path = "models/pocket-tts-en-q8.gguf"
voices_dir = "voices"
lane = "q8"

With a roster, the flat mmproj / asr_workers / tts_lane / tts_voices_dir config keys are not read — each table carries its own — while the same flag on the command line overrides the first table’s value. Two models may not share an id: the second is logged and left out.

Config precedence is defaults < config TOML < explicit CLI flags - unless the TOML carries authoritative = true (what the control page saves), which flips the top: defaults < CLI < authoritative TOML.

Several models serve live from one process through a [[models]] roster instead of the flat model key. Requests route on their "model" field (absent means the default entry). Execution is serialized - one scheduler steps at a time - switches are fast, and every cache level survives a switch: host weights stay mmap’d, each slot keeps its own KV pool and prefix cache, and one model’s GPU state lives in VRAM at a time (the tier drops and re-arms on switch; backend = "cpu" slots never evict the GPU owner). Blank keys inherit the flat defaults; backend is auto | cpu | gpu, and per-entry ctx, quant, kv_dtype, streams, chunk, chunk_idle, page_rows, prefix, mtp, rope_scaling, rope_scale, yarn_orig_ctx, image_mmproj and audio_mmproj override per model:

[[models]]
name = "qwen"      # route id (default: the file's basename)
path = "D:/models/Qwen3.6-35B-A3B-Q8_0.gguf"
default = true

[[models]]
name = "smol"
path = "D:/models/SmolLM2-135M-Instruct-Q8_0.gguf"
backend = "cpu"    # never touches the device - alternating with the GPU slot costs nothing

An explicit --model wins over the roster: the entry with that path becomes the default, and a path the roster does not list joins it as the default. A missing file, or a route name another entry already takes, is logged, and the roster boots unchanged.

Chat and completion requests batch continuously (dasllama/dasllama_scheduler.das): up to --streams generations run concurrently through one eval_batch decode step per tick, with long prompts prefilled in --chunk-token slices, so a new arrival never stalls the running streams for more than one chunk (with no stream running, the slices are --chunk-idle tokens). Requests beyond --streams queue (up to 32; then 503). KV is paged by default - cache memory tracks each stream’s actual context, and finished streams donate their pages to a prefix cache, so a repeated prompt prefix (a shared system prompt, the next turn of the same conversation) attaches instead of re-prefilling. Clients whose connection drops mid-generation are evicted within a tick. Audio uploads queue to long-lived ASR workers and do not block chat generation. OpenAI is stateless — the client resends the full transcript each turn.

7.20.4. Setup mode and the model catalog

A start with no LLM model at all - no --model, no config, or every configured path missing - and no servable --tts or --asr boots into setup mode: the port opens, the control page serves, every inference route answers with a clean error, and the page leads with the model catalog - a curated, sha-pinned list of current models (model_catalog.das, a view over the module’s one model set; commit-pinned HF URLs, canonical sha256, one download at a time, curl -C - resume). A finished download is verified against its pinned sha before it is renamed into models_dir with the .sha sidecar beside it. In setup mode a downloaded entry offers serve this model - the page writes the config and restarts (exit 4, the watchdog contract) into serving it. The same catalog stays available on a serving server for pulling more models.

Catalog entries carry their towers: a vision-capable row offers its pinned mmproj (download, enable vision, restart), a dictation strip under the table offers the ASR tower (parakeet v3), and a speech strip offers the text-to-speech set - the two front-end packs, then one model, or a Pocket file that stands alone. Setup-mode serve this model wires any tower already on disk. Each row wears a fit badge (fits gpu / fits / tight / too big) from the box facts the /catalog document carries; the advertised working set is a hint, not a load gate.

--tts or --asr alone is a serving start, not a setup start: a speech-only server has no LLM slot, so /v1/stats answers the slotless shape with setup false and its speech counters live. An ASR model whose decoder is an LLM - a two-file model, --asr with --mmproj - decodes where the server’s LLMs do: its worker takes the boot’s Metal mode, and /v1/stats names the outcome under asr.models[] as decoder (metal or cpu). A one-file model (whisper, parakeet) decodes on the CPU. A --tts the server cannot serve degrades, it does not die: a missing file or a failed load is logged, the speech route is dropped, and the boot serves whatever is left - the LLM slots if any loaded, else setup mode.

7.20.5. Per-box tuning

This section is the JIT run’s - the SDK path. The standalone download is a fat build: one baked clone of every kernel per CPU class, picked at start, no tuner, no sidecar of its own (only the runtime section its first start writes into ~/.dasllama/tune/); nothing below applies to it, and --tune is ignored there.

The server declares [tune_policy(missing = "restart")]. The first start on an untuned box adopts the shipped profile for its CPU class, and what the profile does not answer runs the dasLLAMA kernel tuner. A tune writes the sidecar and exits with code 3, before the port opens; the watchdog relaunches the server with the winners (without a watchdog, start it again). Thereafter it serves directly and logs the tune status at startup. A start with no model configured skips the tune, so the setup page opens at once. --tune forces a re-tune. DAS_TUNE_POLICY=error skips per-start tuning while developing (it prints the tuner command instead of running it).

The winners live in a sidecar beside the script the JIT runs: utils/dasllama-server/main.tune.json for the server (DAS_TUNE_MANIFEST names another path). dasllama-cli declares [tune_policy(missing = "auto")] - it tunes, then relaunches itself - and keeps its own sidecar beside cli.das. See Kernel tuning for the framework.

The sidecar exchange - dasllama.io’s tune-sidecar service - is closed by default: the three config-only keys exchange_accept (verified | any | off, default off), exchange_submit (ask | always | never, default never) and exchange_url are the only way in. A build without dasLLVM answers 404 on every /exchange* route, and a fat release refuses the levers.

7.20.6. The gpu key

gpu (auto | off | metal | metal-required | vulkan) is the backend selector, and unset behaves as auto: the boot probes the box and serves on the best detected backend - the Metal rails where the box has them, else the Vulkan tier when a device answers, else the CPU - logging one gpu backend auto-detected: line. gpu = off is the explicit opt-out, and DASLLAMA_GPU=0 in the environment keeps its meaning. A model the armed tier cannot serve falls back to the CPU with the reason on the control page.

gpu = vulkan asks for the whole-model resident driver first: a model that fits the card serves entirely from VRAM, and so do its streams - the driver holds a K/V region per stream, the prompt prefills on the device in 512-token windows, the streams decode in one batched step, and a conversation’s next turn picks up its earlier rows from the region they were left in. /v1/stats counts every call the device handed back to the CPU (gpu_cpu_passes); a slot served this way keeps it empty. The control page’s model card says which of these a slot got (served) and why it got less when it did (served_note). A recurrent (deltanet) model serves the same way, its recurrent state kept in the stream’s region beside its rows; a recurrent model with hyper-connections (Qwen3.8-Flash-Next) differs in two ways: a prompt prefills in one piece, and a next turn re-prefills its whole transcript. A slot with a vision or audio tower keeps host-cached sessions, and so does a self-speculative slot where any seat of the round (the draft head, its verify, a rejected round’s rollback) stays on the host. Beneath the driver the per-op rails arm as the fallback for a model that does not fit: expert stacks sized automatically (resident layers fill the VRAM budget, the rest stream) plus the deltanet rail (each deltanet plane in its own format), the attention rail, the dense layers’ attention projections and the shared expert, each over its own planes; a dense layer’s FFN rides the per-op FFN rail (DASLLAMA_GPU_DENSE_FFN=0 keeps it on the CPU). The config keys gpu_layers / gpu_stream are 0 = auto; gpu_dn / gpu_attn / gpu_dense / gpu_vram_mb override the rest, and the DASLLAMA_GPU_* environment variables override everything.

7.20.7. Endpoints

Method	Path	Notes
GET	/	The control page (control.html beside the server): live stats and charts, the models panel (per-slot cards with state and GPU badges, prefix hit rate, activate buttons), the stream swimlane, a chat panel (every sampling knob, thinking inline, mic input, a speaker button under --tts), a speech studio, the config editor with the [[models]] roster and save/restart, GC and drain buttons. Polls /v1/stats and /v1/streams at 1 Hz
GET	/v1/models	Lists every served slot, every TTS model under its id, and the ASR models: the default one as <default slot>-asr beside an LLM, and each under its own id when several serve or no LLM does; requests route on these ids through their "model" field
POST	/v1/models/activate	{"model": name} — make name the default + stepped slot and move the GPU tier to it now (loopback-only; 409 while work is live)
POST	/v1/models/load	{"path", "id"?, "backend"?, "quant"?, "ctx"?, "image_mmproj"?, "audio_mmproj"?, "activate"?} — load a GGUF into a new serving slot with no restart (loopback-only; 409 on a live stream set, a taken id, or a GGUF another slot already serves)
POST	/v1/models/unload	{"model": name} — free the slot’s weights, KV and VRAM; the default slot refuses (loopback-only)
POST	/v1/chat/completions	Chat; stream: true gives SSE, else a buffered reply. OpenAI function calling (tools); image_url parts under --image-mmproj; input_audio parts under --audio-mmproj, or when the image mmproj carries the audio tower (any user message may carry images and clips, up to 16 parts a request, each spliced where the part sits among its message’s text parts; a part is encoded and prefilled once — a continued chat, or the same request again, attaches its span off the prefix cache and never reaches the tower). A stream sent with stream_options: {"include_usage": true} ends, before [DONE], on one chunk with an empty choices list: usage (prompt_tokens, completion_tokens, total_tokens, and prompt_tokens_details.cached_tokens — the prompt tokens the prefix cache attached) and timings (ttft_ms, the scheduler’s admit-to-first-token wall, and gen_ms, first token to finish); a stream that does not ask carries no such chunk
POST	/v1/completions	Raw completion; stream: true gives SSE, else buffered; the same stream_options.include_usage closing chunk as the chat route
POST	/v1/embeddings	L2-normalized sentence embeddings, pooled the way the model’s file asks (pooling_type: last token for Qwen3-Embedding, mean for a chat model); pooling (mean | cls | last) overrides, dimensions keeps the leading components and renormalizes
POST	/v1/audio/transcriptions	Speech to text (multipart upload; needs --asr). One ASR model takes every request, whatever model says; with several, model names one by its id (none, or the <default slot>-asr id: the first; any other name: 404); language defaults to English, and to detection on a model that only detects. response_format=verbose_json adds timed segments
POST	/v1/audio/translations	Speech to English text (needs --asr); model and language are read as the transcription route reads them
POST	/v1/audio/speech	Text to speech (needs --tts): {"model"?, "input", "voice"?, "speed"?, "response_format"?}; model names the TTS model (none: the first; an id no model serves: 404), wav (default) is 16-bit PCM at the model’s rate, pcm the raw samples, the compressed formats answer 400. One synthesis at a time a model, 16 queued
POST	/v1/audio/phonemes	{"model"?, "input", "voice"?} (model as the speech route reads it) gives the normalized text, the dialect the voice speaks (lang) and, per chunk a synthesis would take, its phoneme string — empty for a text-reading model such as Pocket (needs --tts)
GET	/v1/stats	Scheduler counters (gen_tokens, prefill_tokens, TTFT, mtp_drafted / mtp_accepted, the media counters), memory footprint, the hardware line, the ASR and speech worker state, the tts block while a speech model is configured, gpu_cpu_passes, and models[] — one entry per slot with served, served_note, the towers it loaded and its cache counters
GET	/v1/streams	Per-stream states and text tails, the prefix-cache chains, recent ASR jobs
GET	/v1/images	Per-slot prepared-image (.dlim) inventory: source path, mapped flavor, each image’s info
POST	/vad	Silero speech spans over an uploaded clip — the control page’s waveform overlay (120 s at most)
GET	/catalog	The curated model list with local presence, the asr tower row, the tts list, the box memory facts and the download state machine
POST	/catalog/download	{"name": <entry>} — start one catalog download (sha-verified; {"name", "tower": "vision"} / {"tower": "asr"} pull a tower, {"tower": "tts", "file": <file>} one file of the speech set)
GET / POST	/bench	Read the bench state, mode, log and result / start the quiesced benchmark: in process by default (pp512 and tg128 on the served model, then tg128@N on a slot serving two streams or more), or the llama.cpp A/B child when lcpp_bin is configured on a source-tree daslang (POST is loopback-only)
GET / POST	/bake	Read the bake state and log / bake the slot’s prepared .dlim image through dasllama-convert (POST is loopback-only)
GET / POST	/config	The effective config with per-key source (default / cli / toml), the [[models]] roster and the GPU tier status / validate and save an authoritative TOML to the loaded config, or to ~/.dasllama/dasllama-server.toml on a config-less start, applied on the next restart (POST is loopback-only)
GET	/exchange	The tune-sidecar exchange surface: policy, the consent state, the current sidecar’s identity and share state
GET	/exchange/matches	Live lookup of this box against the exchange (a network call — seconds)
POST	/exchange/apply	{"sha": ...} — download, validate and adopt that sidecar, then drain and restart
POST	/exchange/submit	Privacy-strip and submit this box’s own tune to the exchange
POST	/exchange/retune	Arm a local re-tune and restart — the next boot races this box
POST	/exchange/consent	{"accept": true|false} — record the first-contact choice about contacting the exchange; replies {ok, accepted, restarting?}, and an accept on an untuned or stale box drains and restarts the server so the boot resolver runs the lookup
POST	/gc	Schedule a validated collection at the next lifecycle safe point; concurrent requests coalesce
POST	/cache/drop	Loopback-only: drop every slot’s prompt cache (donated pages and checkpoints) while the model stays loaded, so the next request prefills cold; 409 while streams are live
POST	/restart	Drain, then exit 4 — the watchdog relaunches with the saved config
POST	/shutdown	Stop admitting new work, drain, then exit

Unknown endpoints answer 404 through a catch-all that logs the method, the path and the body head; known routes warn per ignored field.

7.20.7.1. Chat

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "Say hello in one word."}],
  "max_tokens": 16, "stream": false, "truncation": "auto"
}'

7.20.7.2. Sampling parameters

Both completion routes accept the OpenAI sampling fields plus the llama.cpp extensions; an absent field keeps the greedy default.

Field	Default	Meaning
temperature	0	<= 0 is greedy argmax
top_k	0 (off)	keep only the k highest logits
top_p	1 (off)	nucleus: keep the smallest probability mass >= top_p
min_p	0 (off)	drop tokens with probability < min_p * max_prob
repeat_penalty	1 (off)	multiplicative repetition penalty over the recent window, once per unique token
presence_penalty	0	flat logit subtraction for every distinct token in the recent window
frequency_penalty	0	per-occurrence logit subtraction
seed	0	non-zero overrides the stream’s RNG seed; identical no-seed requests already reproduce

The cutoffs shape the sampled distribution, so they act only with temperature > 0; the penalties apply in greedy mode too.

7.20.7.3. Thinking and reasoning_content

enable_thinking is tri-state. Absent, the model template’s own default is in force (Qwen3/3.5/3.6, GLM, gemma-4 and gpt-oss templates default on; a bare Qwen Instruct branch defaults off); a present bool - top-level or the llama.cpp spelling "chat_template_kwargs": {"enable_thinking": ...} - overrides it. false on a <think> family appends the template’s empty think block so the model answers directly; false on gemma-4 prefills the closed empty thought channel. A thinking reply’s reasoning span comes back as reasoning_content with content clean of the family’s markers: on the message for buffered requests, and as delta.reasoning_content chunks, streamed before the delta.content chunks, for stream: true. The field is absent when the model did not think.

7.20.7.4. Context truncation

An over-context rendered prompt returns 400. truncation: "auto" keeps the system messages and the tools and drops the oldest complete user-led turns until the prompt plus the requested max_tokens budget fits; when the system, tools and latest turn cannot fit, the request still returns 400. finish_reason: "length" means generation consumed its output budget.

7.20.7.5. Tool / function calling

/v1/chat/completions speaks the OpenAI function-calling protocol: pass tools (and tool_choice: "none" disables, "required" acts as "auto", the forced-function object form is not honored), get back finish_reason: "tool_calls" with message.tool_calls, send the results as role: "tool" messages, repeat. Assistant tool_calls turns and role: "tool" results replay through the chat template on each stateless resend, so agent loops (opencode, pi, …) work end to end. A thinking model that calls tools returns reasoning_content and tool_calls in one response.

The wire format follows the model family: hermes (Qwen2.5 / Qwen3 family - a <tools> system block, <tool_call> JSON), harmony (gpt-oss - developer-turn TypeScript namespace defs, commentary-channel recipient calls), gemma4 (gemma-4 - the declaration / call / response DSL), mistral (v0.3 and the tekken templates - Mistral-Small 3.x, Magistral, Devstral: [AVAILABLE_TOOLS] defs, a [TOOL_CALLS] array) and llama_json (llama-3.x - the whole reply is one {"name","parameters"} object, results on the ipython role). A model whose chat template declares no tool format gets a 400. Streaming with tools puts the reply’s text on the wire as it comes and holds back only the family’s call marker; the parsed calls arrive as one delta.tool_calls chunk at finish; hermes, xml_function and gemma4 hold at their call marker, llama_json at the call object’s opening brace, mistral at the call array’s bracket. A family whose calls ride channels rather than a marker (harmony) is framed whole at the finish.

7.20.7.6. Images

A slot started with --image-mmproj accepts the OpenAI content-parts image form. The URL must be a data: URI - the server never fetches a remote URL:

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,'"$(base64 < cats.jpg | tr -d '\n')"'"}},
    {"type": "text", "text": "What animals are in this picture?"}
  ]}], "max_tokens": 64
}'

The image is decoded and encoded on a dedicated vision worker thread, and its soft-token rows prefill between the two token spans of the rendered turn. One image per request, on the final user message: two images is a 400, an image anywhere else is dropped with a warning. A slot with no vision arm answers 400. An image stream neither reads nor writes the prefix cache, and usage.prompt_tokens counts the soft-token rows as positions. The payload caps at 32 MB of file and 67 MP decoded.

7.20.7.7. Embeddings

input is a string or an array of strings. Each vector is model.config.dim floats from the decoder’s last-layer hidden rows (post-final-norm), L2-normalized, pooled the way the model’s file asks: its pooling_type key names last-token pooling for an embedding model such as Qwen3-Embedding (the row at the appended end token, the way it was trained to be read) and is absent on a chat model, which is read by the mean over positions. pooling ("mean" | "cls" | "last") overrides the file; dimensions keeps the leading components and renormalizes, the Matryoshka read the Qwen3-Embedding family supports. An instruction prefix on the query side (“Instruct: …nQuery: …”) is the client’s text, as the API has no field for it. A chat model used as an embedder gives RAG-grade vectors (good for retrieval / similarity), not a substitute for a dedicated embedding model. See dasLLAMA-09 — Embeddings for the facade side.

curl http://127.0.0.1:8080/v1/embeddings -H 'Content-Type: application/json' -d '{
  "input": ["the quick brown fox", "a lazy dog"]
}'
# -> {"object":"list","data":[{"object":"embedding","embedding":[...],"index":0}, ...],
#     "model":"...","usage":{"prompt_tokens":N,"total_tokens":N}}

7.20.7.8. Transcription (with --asr)

curl http://127.0.0.1:8080/v1/audio/transcriptions \
  -F file=@audio.wav -F response_format=verbose_json

ASR work runs on its own worker threads beside the HTTP tick and the LLM scheduler. Requests beyond the worker count wait in a bounded queue (32 uploads; then 503).

7.20.8. Supervised deployment

The server runs under the shared watchdog in utils/watchdog/, in the release bundle and in a JIT deployment alike. The watchdog needs no arguments: in the bundle it finds the baked exe beside it (watchdog.json names dasllama-server among the bundle’s programs), in a JIT deployment main.das beside bin/Release/daslang.exe. watchdog.json also pins the name, turns the tray on, names tray.ico as its mark, and points the state directory at ~/.dasllama, so the watchdog’s log lands in ~/.dasllama/logs/dasllama-watchdog.log. From the source tree the watchdog does not sit beside the script, so pass --cwd:

bin/Release/watchdog.exe --cwd utils/dasllama-server --jit-stack

The first JIT start on an untuned box writes the tune sidecar and exits with code 3; the watchdog recognizes that bootstrap exit and relaunches. It writes rotating JSON-line logs, polls /v1/models, and after a crash copies the minidump with the matching JIT artifacts, tune manifest and log into ~/.dasllama/logs/crashes/, then restarts with bounded exponential backoff.

daspkg release --root utils/dasllama-server --out <dir> bakes the server into a standalone bundle: the exe, the shared modules and runtime libraries it needs, watchdog beside it, dasllama-cli, dasllama-bench, watchdog.json, control.html and tray.ico. Plain release tunes the kernels on the build box and ships that box’s sidecar; release --fat x86-avx2 (arm-neon on arm64) is what the public download is built from. The config and the tune sidecar live under ~/.dasllama, so an upgrade unpacked over the old bundle keeps them; stop a running server first - Windows locks the DLLs.

7.20.9. Testing

The tests in the tool directory are model-gated and JIT-only: they skip cleanly when the GGUF is absent (set DASLLAMA_MODELS_DIR, one of the knobs in the dasLLAMA knob reference):

bin/daslang -jit dastest/dastest.das -- --test utils/dasllama-server/test_openai_server.das

• test_openai_server.das — endpoint conformance (/v1/models, /v1/embeddings, buffered chat, the tools-unsupported 400, the unknown-endpoint 404) over the real dashv HTTP client, plus model-free parse_tool_calls unit tests; needs tinyllama-1.1b-chat-v1.0.Q8_0.gguf.

• test_openai_server_stream.das — SSE chunk framing, the over-long-prompt 400, two concurrent clients batching on one server, mid-generation disconnect eviction, the prefix cache returning an identical completion for a repeated request; needs SmolLM2-135M-Instruct-Q8_0.gguf.

• test_openai_server_vision.das — the image route end to end on gemma-4-12B with its mmproj and the coco cats jpeg.

• test_openai_server_speech.das — the speech route end to end on a TTS-only boot; needs kitten-nano.gguf and the front-end packs beside it.

• test_openai_server_audio.das and test_openai_server_think.das — the native-audio content parts and the thinking control.

• test_openai_server_mtp.das — the self-speculation default: a NextN-headed slot whose file has no catalog serve_mtp row decodes plain at one stream on Metal, at four and on the CPU, an explicit mtp wins either way, a head-less model with no drafter beside it serves plain; the Metal cells skip on a box with no Metal backend; needs Qwen3.5-0.8B-MTP-Q8_0.gguf and the TinyLlama file.

• test_exchange_client.das, test_model_catalog.das, test_setup_mode.das — model-free: the sidecar exchange client against a fake exchange, the catalog-table invariants, a slotless setup-mode boot.

• test_slot_served.das, test_model_override.das, test_worker_dispatch.das, test_server_flags.das — model-free: every served / served_note arm, the --model-over-roster override, the media worker’s dispatch under the job queue, the flag-presence test behind the config surface’s provenance and the self-speculation default’s rule.

• tests/ — the control page itself under Playwright (Node + chromium), against JSON/SSE fixtures captured from a real server; npm ci && npx playwright test in that directory.

server_bench.das (same directory) measures the serving latencies through the scheduler seam: tok/s, TTFT and inter-token percentiles against batch size, the decode stall per prefill chunk size, and warm against cold TTFT for the prefix cache.

7.20.10. Not yet implemented

On the media path: more than one media clip per request, media on earlier turns of a conversation, and remote image_url fetches.