11. dasLLAMA environment variables
GENERATED from the [EnvConfig] declarations in dasllama/dasllama_env.das - do not edit
by hand. Regenerate with daslang modules/dasLLAMA/harness/gen_env_doc.das; a knob read
anywhere in the tree without a declaration fails tests/test_env_registry.das.
Types: flag is unset-means-default, 0/false/off/no (any case) is false and
anything else true; number falls back to the default when unset or unparseable, with
a logged warning on garbage; text and path are taken verbatim. A SET-BUT-EMPTY
variable counts as unset everywhere. Every knob loads ONCE, at context init, into the
g_env_* globals - hot code reads struct fields, and set_env_variable after startup
is invisible (arm a child process’s environment instead).
11.1. Engine
Read by the inference engine itself, so these affect any program that loads a model.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
flag |
on |
Use the prepared-image file beside a source model for millisecond reloads. 0 skips the file: LLM ggufs take the full parse; audio carriers mint the image in memory and serve that (they are always image-backed). |
|
flag |
on |
Persist the prepared image beside the source; 0 builds it in memory and serves that, leaving no .dlim behind. |
|
flag |
off |
Allow the CPU prefill path even when a GPU prefill override is registered. |
|
text |
unset |
Pin prefill to one registered override by name (e.g. metal, vulkan); anything else declines before upload. |
|
path |
unset |
Split NextN/MTP head GGUF to attach to the trunk being loaded (the split mtp- |
|
text |
unset |
RoPE scaling override for the load: yarn |
|
number |
0 |
The RoPE scaling factor s (the context multiplier) for the override; 0 = the file’s rope.scaling.factor, and yarn needs one. |
|
number |
0 |
YaRN: the original training context the factor extends; 0 = the file’s original_context_length, else its context_length (llama.cpp –yarn-orig-ctx). |
|
number |
0 |
YaRN: the low correction dim beta_fast; 0 = the file’s, else 32. |
|
number |
0 |
YaRN: the high correction dim beta_slow; 0 = the file’s, else 1. |
|
text |
unset |
Pin the matmul backend by name, bypassing the measured auto-selection. |
|
flag |
on |
The streamed split’s host share runs each (expert, row) pair’s whole chain on one lane in one fork a batch - gate, up, act, requant, down, park; 0 = the three region-list GEMV passes with the act and requant between them (the A/B arm). |
|
flag |
on |
The one-pass rare rail runs an expert’s rows through the batched expert cores (the plane read once a tile); 0 = the rows core a row (the A/B arm). |
|
number |
1 |
The rare rail reads a streamed layer’s experts off the tier’s pinned device-layout planes through the format’s row-major kernel - the prompt’s host share and a token’s host picks alike. =1 (the default) takes the rail where the load kept no planar planes (a trimmed lane: one copy of the experts); where the planar planes are loaded (the CUDA home, a lane minted with DASLLAMA_TRIM=0, DASLLAMA_IMAGE=0) the tuned planar kernels serve them. =2 takes the rail on the planar planes too (the A/B arm of the row-major kernels, the environment’s twin of |
|
number |
0 |
The meet hands the host this many of a streamed layer’s rarest experts before the device claims any (0 = none: the two sides claim at their own pace) - a measurement lever, so a carrier the device would take whole still runs the host rail. |
|
text |
auto |
How an IQ3_S weight is served: grid = the disk form (the 3.44-bit grid planes, decoded by a gather), lut = the iq3s4 form (a 4-bit codebook index a superblock, decoded by the LUT kernels - exact, 17% more bytes, the CPU decode several times faster), auto = lut on the CPU rails and the Vulkan tier, grid under the Metal tier (every home serves both forms at every site). |
|
text |
unset |
Pin the batched (prefill) matmul backend independently of the decode one. |
|
flag |
off |
Arm the MoE expert-reuse counter (probes and servers; benches use set_expert_reuse instead). |
|
flag |
off |
Print engine diagnostics (tier selection, upload plan, arm/decline reasons). |
|
flag |
on |
A paged scheduler serving device-home streams keeps their openings in its prefix cache too - a recurrent model’s checkpoints and the finished turns’ rows copied down to host RAM under the cache’s budget, taken back up into a region when a request opens with one the device no longer holds; 0 keeps the regions’ parked claims and checkpoint rows alone - the A/B rail. |
|
number |
2 (the top notch) |
A/B rail for the dispatch caller’s thread priority, -2..2; unset claims the top notch. |
|
flag |
off |
Let the engine dispatch on a job queue nobody configured with setup_dasllama_jobque: a loud warning instead of the panic. A bare queue decodes ~200x slower, so only a run measuring that regime wants it. |
|
flag |
off |
Run every kernel on the calling thread: no dispatch to the job queue at all, whatever queue exists (set_single_thread is the runtime spelling). The browser build’s baseline and the figure that says what the worker pool buys. |
|
flag |
off |
Regenerate the stored parity truth files instead of comparing against them. |
|
flag |
off |
Bucket gguf -> image conversion time by kind over the weight walk; one clock pair per tensor. |
|
flag |
off |
Permit a big gguf load without -jit; the transforms run interpreted, so expect minutes per GB. |
|
flag |
on |
Advisory source-mapping readahead at gguf load (cold-conversion fix); =0 restores on-demand faulting. |
|
flag |
off |
Readahead of the prepared image at its map (madvise WILLNEED, synchronous on macOS): a purged page cache then reads the whole image at the SSD’s rate before the first token - 103 GB in 15 s on the M5 - instead of paying 2.5x on the routed expert stacks over the first 128 tokens, and a WARM map pays ~2.5 s for the page walk in place of 65 ms. A serving process that runs for hours wants it; a bench, a test or a short session does not. |
|
flag |
off |
The streaming silero VAD ( |
|
flag |
off |
One switch for the measured-best GPU rail set; any DASLLAMA_GPU_* knob still overrides individually. On a Vulkan build it is also the want the tower driver reads before any device init: the vision and audio towers’ block loops serve on the device only while it is on, and decline |
|
flag |
off |
The environment’s spelling of the GPU home (the program’s is |
|
number |
-1 (auto) |
How many MoE expert layers to hold resident on the GPU; -1 lets the upload walk place the split. |
|
number |
-1 (auto) |
How many MoE layers to stream rather than hold resident; -1 is auto. |
|
number |
probed |
Override the detected VRAM budget in MiB that sizes the resident expert stacks; a pinned cap takes no headroom and reads no OS room. |
|
number |
built-in floor |
Lower the context floor for arming the resident decode driver, for a short-context session on a small card. |
|
number |
the model’s context |
Cap the resident driver’s KV mirror at this many positions (the card’s room and the 4 GiB binding cap on the widest layer’s rows still bound it); a cap under the arming floor lowers the floor to the cap (never under 256), since the cap is the shape the caller serves; 0 follows the model’s context; a server keeps the model’s. |
|
flag |
off |
The context pin (DASLLAMA_GPU_CTX_MAX, set_gpu_ctx_max) is the context asked, not a cap: a K/V mirror the card cannot hold at it declines the resident driver instead of shortening, and the per-op rails serve the asked context; the load log names the room. |
|
number |
0 |
The whole-model driver’s prompt window in rows: the prefill panels’ size, and the rows a hyper-connection model’s host expert step and the hot pool’s window chain take at once; a longer prompt runs as sequential windows, each paying every routed layer’s expert pass again. 0 (the default) lets the driver pick: 4096 where its plan fits with the wider window’s scratch, else 2048. Rounded down to a multiple of 128, never under 128; the plan charges the panels at this count, so a wider window shortens the K/V mirror or the pool. |
|
flag |
on |
Stream each routed layer’s expert stack to the device for the prompt, where the routed experts serve on the host (a hyper-connection model, or a plain MoE’s host layers) - on the Vulkan tier over its transfer queue, on the CUDA home over its copy stream: pinned device-layout mirrors of every routed layer at load, a device slot (a layer’s stack, or under DASLLAMA_GPU_MOE_SPLIT the per-expert chain’s ring of experts), the copies landing while the chain computes, the host summing what the meet hands it. 0 keeps the host’s sums and the hot pool’s window chain. |
|
number |
0 (every host layer) |
Under DASLLAMA_GPU_PF_STREAM, mirror only the first this many host layers’ expert stacks; the rest keep the host’s sums. A parity cell’s lever: a few layers’ mirrors beside a planar load, where every layer’s would not fit the box. |
|
number |
0 (the plan’s own count) |
Serve at least this many routed layers’ experts on the host - the first in layer order - on the whole-model driver of any MoE that routes no more picks a row than the split command serves, the gemma-4 MoE with its parallel dense expert included, whatever the card holds (set_gpu_host_layers in process): the host-layers form, with its split token command and cut window chain, reached on any card. A hyper-connection model serves every routed layer on the host regardless. |
|
text |
unset |
The K/V codec the whole-model driver’s mirror is asked to arm at, where no caller set one in process (set_gpu_kv_dtype): f16 (the default), q8_0 (half the mirror’s bytes) or tq4 (a quarter). Sessions of the armed codec are served; any other spelling reads as f16. |
|
flag |
on |
The whole-model resident driver for a model that fits the card, MoE included; 0 keeps the per-op rails (the A/B lever). |
|
flag |
on |
A sliding-window layer’s K/V mirror rows on a GPU home hold its window plus the rows one command writes, a position’s row at its slot of that ring; 0 keeps every position’s row (the flat slab, the A/B lever). A ringed history serves a continuation that starts no more than that slack under the rows it holds; one further back prefills again. |
|
flag |
follows DASLLAMA_GPU |
DeltaNet rail on the GPU. |
|
flag |
follows DASLLAMA_GPU |
DeltaNet decode rail on the GPU. |
|
flag |
follows DASLLAMA_GPU |
Attention rail on the GPU. |
|
flag |
follows DASLLAMA_GPU |
Dense-weight rail on the GPU. |
|
flag |
on |
The dense rail’s attention half (per-layer q/o pairs); opt-OUT, measured a loss on gemma-4. |
|
flag |
off |
The dense rail’s gemma-4 shared-expert w1/w3/w2 triple; opt-IN, measured a wash-to-loss. |
|
flag |
on |
The dense rail’s FFN half (each dense layer’s w1/w3/w2 triple, the decode row’s FFN on the device); 0 keeps a dense layer’s FFN on the CPU (the A/B rail). |
|
flag |
follows DASLLAMA_GPU |
Shared-expert rail on the GPU. |
|
flag |
off |
Fused QKV rail on the GPU; stays off under DASLLAMA_GPU (measured a wash). |
|
flag |
on |
Keep the classifier plane on the GPU - the best ms-per-GB region; 0 is the CPU A/B lever. |
|
flag |
on |
Device-side routed MoE combine; 0 falls back to the host combine. |
|
flag |
on |
Split each STREAMED expert layer’s prefill between the GPU (the experts it can stream in the CPU’s time) and the CPU (the rest, concurrently); 0 streams whole layers. |
|
flag |
off |
A decode token’s picks on a streamed layer take the prefill’s route: the meet claims them as the GPU’s per-expert DMAs from the busy end and the CPU’s rail from the rare end, in place of the one-row host step. The experiment knob; the window floors lift under it. |
|
flag |
on |
The hot expert pool fills by DMA: a token’s miss is copied off its pinned stack into the pool’s coldest unpicked slot on the home’s copy queue (the Vulkan tier’s transfer queue, the CUDA home’s copy stream), every miss a step and no host time, so the next token finds it; and a prompt’s streamed window serves the pool’s hits off their slots first and lands its busiest misses in the pool’s free and coldest slots, so the decode behind the prompt starts warm. 0 keeps the host’s gather-and-upload fill under its step budget and the window off the pool. The A/B knob. |
|
number |
0 |
The share of a decode step’s pool misses the DMA fill admits past the heat gate - the warmest first, each into the coldest unpicked slot, landing for the next step while this step’s host sums them - so the pool’s working set follows the stream instead of the gate’s hot few. 0 keeps the heat gate alone; 1 admits every miss the tier can copy. Needs DASLLAMA_GPU_HOT_DMA. |
|
flag |
on |
Decode FFN tail: a resident expert layer’s FFN carries the combine, the residual and the next layer’s attention feed on the device, so the two submits share one fence wait; 0 collects every FFN on the host. |
|
flag |
on |
Whole-token decode span: every resident expert layer’s attention, router, top-k, FFN and combine as ONE recorded chain and one submit per token; 0 runs the per-layer submits. |
|
number |
0 |
Hot expert slots a layer. On the per-op tier: hold the N hottest experts of a streamed layer resident (unset = none). On the whole-model driver with the routed experts on the host (a hyper-connection MoE): the slots of the hot expert pool - unset = every slot that fits beside the weights and the mirror, else up to 32 out of the mirror’s context; a count asks that many; 0 keeps every routed expert on the host. |
|
flag |
off |
Report lifetime GPU queue submissions (real commands plus staging round-trips), and trace every routed row the host sums - the split token command’s host segment and the CPU forward alike - as one moe_k line (the layer, the session, the first pick, the L1 of the feed row and of the routed sum). |
|
text |
unset |
Record the routed picks of every layer’s host expert step under the whole-model driver - a layer, its rows, k expert ids and weights a row, in serving order - and write them to this file when the model drops: the hot expert pool’s policy experiments replay the trace offline (harness/hot_pool_sim.das). Unset: no trace. |
11.2. Vision
The image preprocessing rail: dynamic-resolution token bounds and the debug dump.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
number |
40 (the gemma-4 model default) |
Minimum soft tokens per image; dynamic-resolution geometry upscales to reach it. |
|
number |
280 (the gemma-4 model default) |
Maximum soft tokens per image; dynamic-resolution geometry downscales to fit. |
|
text |
unset |
Path prefix: dump every letterboxed image fed to a vision embedder as |
|
flag |
on |
Serve the image turn as ONE fused prefill (per-query span mask); 0 pins the three-eval splice (head / media rows / tail) - the parity A/B rail. |
11.3. Metal backend
Apple GPU backend. Absent on non-Apple builds, where setting them does nothing.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
path |
unset |
Directory to write each kernel’s generated source as |
|
flag |
on |
Produce logits on the GPU; 0 pulls the classifier back to the CPU. Blob-only models force it on. |
|
flag |
on |
Tiled QK/AV prefill attention (~10x the trio GEMMs); 0 pins the trio. |
|
flag |
on |
Serve the non-causal media span on the prefill driver (AttnArgs.uend); 0 declines it to the CPU arm, which then needs declared CPU intent. |
|
flag |
on |
Serve tower/embedder encodes on the Metal tower driver - the gemma4uv chain, the gemma4v ViT and gemma3v SigLIP block loops (their exact-plane lanes), the whisper-class block loops, the gemma4a Conformer and the canary / parakeet FastConformer block loops (their f32 lanes), the whole StyleTTS2 (kitten / kokoro) synthesis - PL-BERT, the text and duration encoders, the duration head, prosody, the decoder, the harmonic source, the generator and the inverse STFT - on both weight lanes, the Pocket TTS codec and frame loop, the conv frontends (whisper im2col + the qwen3a conv2d, which serves the q8 default), the GigaAM encode and CTC head (all three weight lanes: the f16 halfwords, the q8 slab, the kq lane’s own planes), and the silero VAD seat; 0 loads each family’s CPU image, the lane whole on the CPU. |
|
flag |
on |
Serve a tower’s weighted GEMMs from a halfword twin of its weights on the crowned tensor mul_mm twins: the image’s baked f16 twin plane where the mint bakes one (only for f16-sourced GEMMs, so it holds the file’s own values), and the FastConformer chain’s device twin of parakeet’s f32 blob. Needs a tensor crown - a box without one, or an image without the plane, keeps the f32 route; 0 pins the f32 GEMM route - the A/B rail. |
|
flag |
on |
Serve head-72 tower attention at %64 canvases on the lifted flash kernel (no score slab, no rowstat pass, no head restrides - the gemma3v-geometry ViTs); 0 pins the three-pass slab - the A/B rail. |
|
flag |
on |
Serve the whisper decoder side (cross-KV; the decode step under wdec_step) on the Metal ASR-decoder driver; 0 loads the lane’s CPU image, the whole lane on the CPU. |
|
flag |
on |
Serve the whisper DECODE STEP on the GPU too (needs wdec); 0 loads the lane’s CPU image, the whole lane on the CPU. |
|
flag |
on |
Bake a whisper decoder narrower than 512 (tiny) into its Metal image with the decode STEP on the CPU - the device step’s per-dispatch latency loses at that width; cross-KV stays on the device; that image is its own file ( |
|
flag |
on |
Serve the transducer greedy decode (parakeet, GigaAM’s RNNT head) on the Metal ASR-decoder driver: the joint’s encoder projection off the encode’s device rows and the whole loop in command buffers of rounds, the tokens read back once; 0 pins the CPU decoder on a CPU image - the A/B rail - while a Metal image’s decode refuses: the image never runs on the CPU. |
|
flag |
on |
Fused single-pass decode attention (a head of 64, 96, 128, 256 or 512 on every KV codec; sliding windows, attention sinks, the logit soft cap and a second head class included), on the single row, the speculative verify’s rows and the batch; 0 is the A/B rail to the chunked pair. |
|
number |
64 |
Row count below which attention uses the single-chunk kernel; clamped to 128. |
|
flag |
on |
The mul_mm prefill GEMM; 0 falls back to the legacy per-op path. |
|
flag |
on |
GEMV-tail prefill dispatch: npos % 32 in [1,8] peels the last M tile’s real rows onto the fixed-B GEMV family instead of billing a full 32-row tile; 0 pins the padded-tile path (the A/B rail). |
|
flag |
on |
The K-quant rows sites (two to eight rows: the verify, the batch step, the prefill’s short window) read x as the f16 twin the row producer stores beside it, for every format whose half-x route the box crowned (kq_rows_half_ |
|
flag |
on |
Apply the box profile’s tensor-lane crowns (its runtime.metal_tensor set), so every family the box’s race crowned serves its tensor twin; 0 leaves the set empty and every family on its base form past the routed twins’ knee ( |
|
flag |
off |
The dense Q8_0 and K-quant prefill sites read a stored f16 converted copy of their planes (the dev-W panels the image bakes, or the resident ones); the A/B rail of the default - per-window scratch panels past each site’s row floor, its own bytes under it - which the image lane name carries as -devw. |
|
flag |
on |
Fused prefill qk_norm+rope pass (one panel rewrite, q panel dual-stores its f16 twin); 0 restores the split norm/rope/cvt dispatches (the A/B rail). |
|
flag |
off |
1-D-grid tall hmm GEMM twin: the threadgroup launch order is forced linear (M-fastest) instead of the driver’s 2-D walk; experiment rail, off by default. |
|
flag |
on |
Last-layer FFN tail narrowing: the final dense layer’s FFN runs the last row alone (only the classifier reads past the final attention); 0 pins the full-panel tail (the A/B rail). Session keep_hidden opts a forward out. |
|
flag |
on |
Double-buffered W staging on the dense staged prefill GEMMs (q8 and kq stamps; the MoE family measured a loss and stays single-tile): two ping-pong tiles, one barrier per 64-chunk, staging overlaps the tensor op; 0 pins the single-tile forms (the A/B rail). |
|
flag |
on |
Split-K on the short window’s prefill tiles (q8, every dense K-quant, grid and per-32 format, bf16): a 32-row tile grid of at most half the sidecar’s metal_split_k_tgs threadgroups divides its K across up to eight threadgroups a tile, the partial planes summed by one reduce dispatch; 0 pins the whole-K tiles (the A/B rail). |
|
number |
~4 layers/chunk |
Command-buffer split: each chunk commits as encoded so the scheduler overlaps chunk k with k-1. |
|
flag |
off |
Skip per-dispatch retain/release on the prefill command buffers. |
|
number |
0 (off) |
MB budget for RESIDENT dev-W f16 panels under DASLLAMA_METAL_DEVW=1: static weights dequantize ONCE and the panels persist across forwards (vs the per-forward scratch re-dequant). Sites past the budget fall back to scratch. |
|
flag |
on |
Record each prefill chunk as a step graph and replay on a concurrent encoder; 0 is serial-encode rollback. |
|
number |
1 |
Graph scheduling mode for replay; 0 keeps capture order. |
|
flag |
on |
Concurrent encoder for the single-stream decode step, with barriers only at detected hazards. |
|
flag |
on |
Every decode sandwich seam as one residual step (the branch’s post norm, the add, the next norm); 0 restores the split chain for A/B. |
|
number |
-1 (adaptive) |
The single-row Metal decode’s greedy chain (the next step pre-encoded on the GPU’s own argmax): 0 off, 1 forced, -1 adaptive (backs off after a miss). A session sampled at temp > 0 never chains, whatever the mode. |
|
number |
4 |
Rows per MoE router/expert GEMV threadgroup, clamped 1..32 (the dense GEMV geometry is fixed). |
|
flag |
on |
The single-row decode’s step wait spins on the command buffer’s GPU end time before the blocking wait: the calling thread owns the step and a blocking wake returns late on a box whose cores the jobque lanes fill (M4 Pro: 0.35 ms per token, a step every third step); 0 blocks in waitUntilCompleted alone - the A/B rail. |
|
flag |
on |
Single-pass B8 twin for K-quant small-batch mv at B=5..8; 0 is the A/B rail. |
|
number |
4096 |
The device-side KV mirror’s idle budget in MiB: the bytes idle mirrors evict down to once a step is served. A live step’s demand is bounded by the device’s largest buffer alone (unified memory: what fits, serves), and a step past that takes the CPU rails with one logged line. |
|
flag |
on |
A released paged session’s KV mirror goes to the next session whose leading pages are the same pool groups, its rows not uploaded again; 0 uploads every attached row - the A/B rail. |
|
flag |
on |
An f16-KV prefill writes its K/V rows into the session’s KV mirror and its attention reads the cached rows there (the device pair, at any row count), so a continuation gathers nothing on the CPU and the first decode step uploads nothing; without the attn_dev crown the attention keeps its f32 panels and the gather, and the whole panel still converts into the mirror, so that first step uploads nothing there either; 0 keeps the panels and leaves the mirror to the decode step’s upload - the A/B rail. |
|
flag |
on |
Pin the served working set (weight regions, planes, the recycled pool buffers) in an MTLResidencySet, macOS 15+: the per-commit residency pass stays a no-op, so the first submission after a CPU-only window stops repaying it (~15 ms/encode on the M1 Max released-rig image cell; 0 is the A/B rail. |
|
number |
180 |
Seconds the residency-set heartbeat keeps re-requesting residency after the last served step (a requested set still un-wires over a long CPU-only window; 0 disables the heartbeat - the A/B rail. |
|
flag |
off |
Barrier at every dispatch instead of at tracker-detected hazards (correctness bisect). |
|
flag |
off |
Treat every detected hazard as strict, widening barriers (correctness bisect). |
|
flag |
off |
Per-step pipeline trace: GPU envelope and true inter-step handoff idle, first steps plus outliers. |
|
text |
unset |
Comma-separated dispatch names to skip in decode - a bring-up bisect that breaks correctness. |
|
text |
unset |
Prefill twin of DASLLAMA_METAL_DECODE_SKIP. |
|
text |
unset |
MTP bring-up bisect: cold logs why the Metal seats left a round to the plain step (the forced reject is set_mtp_force_reject_every). |
|
number |
5 |
Batch size at which batched decode switches from GEMV to GEMM; floor 2. |
|
flag |
on |
Split-K in the batched decode GEMM. |
|
number |
4 |
Command buffers per batched decode step, clamped 1..16. |
|
flag |
on |
Concurrent encoder for batched decode. |
|
flag |
on |
Epilogue fusion in batched decode. |
|
flag |
on |
Fixed-B GEMV forms for small batched decode. |
|
flag |
on |
The column-tiling MV8 probe for batched decode. |
|
flag |
off |
Pipelined batched decode submission. |
|
flag |
off |
The batched decode pre-encodes the next step under the current one’s GPU run and pokes its token inputs at commit. Off by default: it times the same as building at the call and holds a second step’s buffers. |
|
flag |
off |
The mul_mm rail for batched decode; measured negative at M-pad-32, kept as the A/B rail. |
11.4. Vulkan backend
Vulkan GPU backend. Present only where the dasVulkan package is installed.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
text |
auto |
Cooperative-matrix mode (auto = cm2 where the device has NV_cooperative_matrix2, else mm, else sdot4; |
|
text |
32 |
Small-batch tier: 32 = sdot4 (default, beats both coopmat tiles below the crossover), 64 = coopmat M, 128 = always-L. |
|
number |
64 |
Small-d cutoff routing narrow roles (k/v) to the small tier; widening measured worse, so this is an instrument. |
|
flag |
on |
Fused add+rms+requant on every site of the decode token command and the prefill’s batch pairs, plus the fused qk-norm+rope; 0 pins the split dispatches for a same-build A/B. |
|
number |
0 |
The whole-model driver’s greedy chain: the next one-row step submitted behind the running one on the device’s own pick; 0 off, 1 every greedy step, -1 adaptive (backs off after a miss). A sampled session never chains. |
|
flag |
on |
The resident prefill’s FFN GEMMs (gate, up, down) take the f16 activation feed where the tile family admits it; 0 pins the FFN group to the Q8 activation blocks the CPU chain quantizes, while the attention group keeps its own pick - the A/B rail that takes the feed’s rounding out of a resident-vs-CPU compare (gemma’s logits move with it). |
|
flag |
on |
Stream expert uploads on the dedicated transfer queue, overlapped via a timeline semaphore, and land the resident token command’s logits there behind a compute-timeline wait; 0 keeps the single-queue rail, where the token command copies its own logits under its fence. |
|
flag |
on |
Run the cm2 tiles’ four-wide decode callback where the device has VK_NV_cooperative_matrix_decode_vector; 0 strips it and serves the scalar callback - the same-build A/B and the fallback probe. |
|
flag |
on |
The resident prefill window’s per-layer-embedding gate and projection serve a file’s f32 rows on the staged KHR tile (the rows’ words rounded to f16 on the tile’s stage ahead of the KHR multiply, at subgroup 32); 0 keeps them on the scalar f32 tile - the same-build A/B. |
|
flag |
on |
Stream mirrors import the mapped .dlim (VK_EXT_external_memory_host) instead of pinned copies, on the copy arm alone ( |
|
flag |
on |
A streamed stack’s planes are read off the .dlim file straight into its pinned memory, the mapping’s expert pages never faulted; =0 copies them out of the mapping (the A/B arm). |
|
number |
0 |
The expert cache’s pinned budget in MiB for the streamed layers’ expert stacks: the resident slots of every streamed layer and the transient ring together; 0 holds every streamed expert resident (the whole stacks in pinned memory, as a box with the RAM does). Under it each streamed layer keeps a quota of its experts resident and the rest are read off the lane file on demand into the ring, through the format’s row-major kernels on the host and the DMA on the device alike; needs a mapped lane (a chunk-built image has no file to read from). |
|
number |
0 |
The expert cache’s transient ring in experts a plane kind (gate, up, down): the slots the on-demand reads land in, freed at a window’s or a token’s end. 0 sizes it to the largest streamed layer’s non-resident count, so one window’s misses always fit. |
|
number |
8 |
The expert cache’s reader threads: the on-demand reads off the lane in flight at once, each thread one read at a time on its own file handle. 0 reads on the calling thread, one at a time. |
|
number |
0 |
The experts one bundle of the streamed expert chain takes at most (0 = a sixteenth of the layer’s experts, 1 to 32). A bundle is the unit the device claims at the meet, at most two in flight, so the cap sets the device’s pace against the host’s share; the A/B lever. |
|
flag |
on |
Mint the TRIMMED GPU lane - the armed home’s, vulkan or cuda (the default; =0 mints the lane with its planar families): the device twin plus the packed emb region the CPU embed reads, the planar weight families dropped - a MoE’s expert stacks too, the device holding them or the host reading its streamed layers’ pinned mirrors through the rare rail (so the prompt stream and the rail’s arena arm must be on where routed layers serve on the host). A model the whole-model driver would not take whole keeps its families, and the load says so, as does one with per-layer embeddings, biased expert stacks or the gemma-4 MoE form (the host reads their planar planes); a call the driver passes to the CPU rails panics by name. Folded into the lane’s identity, so a flip re-bakes the lane; |
|
flag |
off |
Tag every device allocation priority 1.0 (VK_EXT_memory_priority). Off by default: on the NVIDIA WDDM driver the tag makes the tagged weights the pageable set, and a resident model then decodes at PCIe speed once anything is demoted. |
|
flag |
on |
Vulkan flash attention: the decode fa kernel pick AND the cm2 prefill fa tile; 0 falls back to the chunked/scalar paths. |
|
flag |
on |
Serve the vision towers’ block loops (gemma4v, gemma3v, qwen3v, qwen25v), the audio towers’ (the whisper-class towers, gemma4a, canary) and the TTS seats (the StyleTTS2 families kitten and kokoro, Pocket) on the Vulkan tower and TTS drivers - the announce is |
|
flag |
off |
GigaAM’s lane policy on the Vulkan tower: 1 prefers the exact lane - the block GEMMs on the f16 GEMM over the carrier’s own halfwords - over the q8 image for an un-pinned load; 0 (the default, until the two lanes are timed on a Vulkan box) keeps the q8 image. Both lanes serve on the device on the f16 GEMM feed alone (off it, every lane loads the CPU image); the family’s lane pin overrides it, |
|
flag |
on |
Serve the whisper decoder side (the per-window cross-KV and the decode step) on the Vulkan ASR-decoder driver - the announce is |
|
flag |
on |
Serve the transducer greedy decode (parakeet, GigaAM’s RNNT head) on the Vulkan ASR-decoder driver: the joint’s encoder projection off the encode’s device rows where the lane’s projection is f32, and the whole loop in command buffers of rounds, the tokens read back once; 0 pins the CPU decoder on a CPU image - the A/B rail - while a Vulkan image’s decode refuses: the image never runs on the CPU. |
|
flag |
on |
Merged k |
|
flag |
on |
The last layer’s FFN runs on the window’s last 32 rows only (the classifier reads one); 0 runs it over the whole window for a same-build A/B. |
|
flag |
on |
Prefill record/execute overlap: the window chain submits in ramped chunks (1,2,4,8 layers) so the GPU starts while the CPU still records; 0 pins the single fenced submit (the per-role GPU profile pins it too, so a chunk gap never bills to a role). |
|
number |
0 |
Layers one prefill submit holds at most (the overlap ramp’s cap, 1..8); 0 = 1 on an integrated GPU, 8 on a discrete one. A Linux amdgpu resets the device under a job past its watchdog (about 10 s), which an iGPU’s 8-layer chunk reaches on a long prompt. |
|
flag |
on |
Device-side token-embedding gather for the resident prefill and every token command (the one-row step, the N-row command, the NextN verify and draft): a q8 or K-quant table off its own plane (the tied classifier’s, or an untied table’s where it fits the room past the weights and the K/V), an F32/F16/BF16 table off its exact f32 rows uploaded under a 512 MB cap; 0 keeps the CPU embed loop. |
|
flag |
off |
Pin REQUIRE_FULL_SUBGROUPS on every class pipeline (instrument; measured slower than plain pipelines on the mm_a gate shape, so those are the default). |
|
number |
32 |
The subgroup width every class pipeline pins where the device’s own differs and its compute stage may pin it (Vulkan 1.3 subgroupSizeControl; RDNA runs 32 or 64 lanes and defaults to 64): the coopmat tiles, the h128 attention twin and the tower tiles own four 32-lane fragments a workgroup, so a device left at 64 serves the sdot4 tiles and the scalar attention instead. 0 = the device’s own width. |
|
flag |
off |
Place the planes in the ReBAR device-local host-visible memory type (one larger than 1 GB present) and write uploads straight into them. Off by default: on the NVIDIA WDDM driver a mapped plane is a host-side shadow the manager parks in system memory under pressure, and a host-experts plan then decodes at PCIe speed (gemma-4 26B Q8_0 8.1 tok/s mapped against 32.7 staged, the 35B-A3B 38 against 71: debug-jit readings, the ReBAR entry in modules/dasLLAMA/PERF_LEDGER.md) - the default stages every upload into unmapped device-local planes. |
|
number |
0 |
Arm the resident driver with f32 KV mirrors instead of the f16 default (A/B instrument; only sessions of the armed codec are served). |
|
number |
0 |
|
|
number |
0 |
Keeps a fused form of the token command on its split dispatches (a same-build A/B; the logits stay right; the N-row command shares the gate-up and the fused q/k norm through the layer flags read at load) - bits 1 = the wo GEMV’s residual epilogue, 2 = the down GEMV’s, 4 = the fused gate-up twin, 8 = the prologue’s add+rms+requant stamp, 16 = the post-attention one (the wo epilogue too), 32 = the post-FFN one (the down epilogue too), 64 = the fused qk-norm+rope; the load log names each bit as taken or inert on the model |
|
number |
0 |
cm2 prefill tile pick: 0 = the wave model’s pick ( |
|
number |
0 |
cm2 split-k: 0 = the pick’s own (the Vulkan wave model |
|
number |
1 |
The CUDA home’s window chain on the MMQ GEMMs: the int8 one ( |
|
number |
1 |
The CUDA home’s launches under programmatic dependent launch: a kernel is launched while the one before it in the stream still runs, and its entry wait ( |
|
number |
1 |
The window chain’s deltanet scan on the discrete homes: 1 = the chunked form on the 16 x 16 cooperative tiles ( |
|
number |
0 |
The CUDA home’s greedy chain: the next one-row step launched behind the running one on the device’s own pick; 0 off, 1 every greedy step, -1 adaptive (backs off after a miss). A sampled session never chains. |
|
path |
unset |
Directory of |
|
path |
unset |
Directory to write each kernel’s emitted words as |
|
flag |
off |
Barrier at every dispatch (correctness bisect). |
|
flag |
off |
Log every detected hazard and the barrier it produced. |
11.5. MoltenVK (macOS)
MoltenVK’s own configuration - listed because the vulkan tier arms it on Apple.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
flag |
unset (tier forces 0) |
MoltenVK’s own knob. Its argument-buffers default miscompiles the batch tile kernels (stores never land, silently); the tier forces discrete bindings when this is unset. |
11.6. Accelerate / AMX backend
Apple Accelerate / AMX float lane. DASLLAMA_ACCEL arms the whole group.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
flag |
off |
Arm the Apple Accelerate / AMX float-batch override. |
|
flag |
off |
Use the BNNS f16 lane inside the Accelerate backend. |
|
number |
0 |
Strip count for the f16 lane; 0 lets the backend choose. |
|
number |
d/256, max 8 |
Strip count for the f32 sgemm lane; 0 keeps the shape-derived default. |
|
number |
backend default |
Minimum MMAC count below which Accelerate declines and the daslang kernel runs. |
|
number |
32 |
Minimum token count for the Accelerate float-batch override, floor 1. |
11.7. Harness - tuner and probes
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
path |
unset |
Directory holding the .gguf models the probes, benches and tests load. dasllama-server’s model catalog downloads here too when set. |
|
path |
auto-resolved from the models dir |
Model used by the tuner’s confirm gate (FULL path, not a bare filename). Unset: the gate auto-resolves from the models dir - the preferred confirm carrier, else the largest present q8 gguf; the fallback pins only when the box has no q8 model at all. |
|
path |
auto-resolved from the models dir |
IQ2_XXS vehicle for the tuner’s serving-crown confirm (FULL path). Unset: the confirm auto-resolves the first IQ2_XXS.gguf in the models dir; none found = no crown (the base kernel serves). |
|
path |
auto-resolved from the models dir |
gemma-4 vehicle for the tuner’s assistant-depth confirm (FULL path; its mtp- head must sit beside it). Unset: the confirm auto-resolves the first gemma-4*.gguf with a head in the models dir; none found = depth 1 serves |
|
text |
unset |
Override the batched-dispatch chunk count in the 1-core GEMM probe. |
|
text |
Qwen3VL-8B-Instruct-Q8_0.gguf |
Model filename (inside the models dir) the image-turn attribution probe loads. |
|
number |
unset |
Use the 2D batch grid in the parity probe. |
|
text |
unset |
Restrict the 1-core GEMM probe to one backend. |
|
text |
unset |
Restrict the 1-core GEMM probe to one shape. |
|
number |
unset |
Restrict the 1-core GEMM probe to one token count. |
|
flag |
off |
Run the 1-core GEMM probe through the fork pool rather than inline. |
|
flag |
off |
Compile the generated tune probe without running it. |
|
flag |
off |
Compile the smmla probe without running it. |
|
flag |
off |
Run the Accelerate contention probe against the Metal backend. |
|
flag |
off |
Per-job dump in the dispatch probe. |
|
number |
unset |
Synthetic per-job spin in nanoseconds for the dispatch probe. |
|
flag |
off |
Exercise batched dispatch in the dispatch probe. |
|
flag |
off |
Exercise the two-wave dispatch shape in the dispatch probe. |
|
flag |
off |
Skip restartHeaps on pooled fork-context reuse (A/B for the fork-pool heap cost). |
|
number |
unset |
Worker spin-before-park window in microseconds, applied via set_jobque_spin_us. |
|
number |
unset |
Join-side spin window for the 1-core GEMM probe. |
11.8. Benchmarks
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
flag |
off |
Let a bench cell measure on fallback/absent tune winners, and skip the auto-tune mint entirely (untuned kernels keep fallback stamps instead of paying the minutes-long tune). The gate exists because a stale sidecar silently downgrades every kernel and the whole board sandbags; only set for dev runs whose numbers are never published. |
|
text |
tinyllama |
Model name for the isolated GEMM bench. |
|
number |
unset |
Token count for the isolated GEMM bench. |
|
flag |
off |
Skip the row-major arm of the isolated GEMM bench. |
|
flag |
off |
Per-run detail from the comparison bench (lcpp_bench). |
|
number |
unset |
Override the minimum rows per dispatch chunk in the GEMM bench. |
|
flag |
unset |
Exercise the fused decode arm in the emission bench. |
|
number |
3 |
Timing rounds per cell in the Metal kernel labs. |
|
number |
4 |
Passes per round in the Metal MoE lab. |
|
number |
16 |
Simdgroups per threadgroup in the Metal attention lab. |
|
text |
unset |
Dump generated MSL from the Metal labs instead of running it. |
|
text |
unset |
Substring filter restricting which arms the Metal MoE lab runs. |
|
text |
unset |
Substring filter restricting which shapes the Metal GEMV/GEMM labs run. |
|
text |
unset |
Substring filter restricting which kernel variants the Metal GEMV/GEMM labs run. |
|
path |
_wcliff.bin |
Scratch file the write-cliff probe writes, cwd-relative by default; point it at the drive under test. |
|
number |
50 |
Gigabytes the write-cliff probe writes before reporting. |
11.9. Profiling and baselines
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
path |
bin/daslang |
daslang binary the profiling harness shells out to. |
|
text |
probed |
Override the box identifier recorded in a profile. |
|
text |
probed |
Override the CPU name recorded in a profile. |
|
text |
probed |
Override the OS name recorded in a profile. |
|
number |
probed |
Override the RAM size in GiB recorded in a profile. |
|
number |
probed |
Override the thread count recorded in a profile. |
|
text |
probed |
Override the GPU name recorded in a box profile. |
|
path |
unset |
Path to the llama-bench binary used to establish reference baselines. |
|
path |
unset |
Path to a clean-build llama-bench for the reference column. |
|
path |
unset |
Path to a stock-build llama-bench for the reference column; its sibling llama-mtmd-cli serves as the metal/amx image-chat reference. |
|
path |
unset |
Path to a Vulkan-build llama-bench (setup_lcpp_ref’s vulkan flavor) for the vulkan leg’s reference column. |
|
path |
unset |
Path to a CUDA-build llama-bench (-DGGML_CUDA=ON at the reference tag) for the cuda leg’s reference column. |
|
path |
../whisper.cpp |
whisper.cpp checkout used for ASR reference baselines. |
|
path |
unset |
Directory of whisper.cpp models for ASR reference baselines. |
|
path |
../CrispASR |
CrispASR checkout used for the GigaAM ASR reference baselines (its Metal build in build/, its CPU-only build in build-cpu/). |
|
path |
../translations |
Mozilla translations checkout whose patched inference/build/marian-decoder is the Bergamot translation reference. |
|
path |
unset |
llama-mtmd-cli binary for multimodal reference baselines. |
|
path |
unset |
Base CPython 3.10-3.12 for creating the ASR oracle venvs, when neither PATH names nor uv resolve one in range. |
|
path |
unset |
Python interpreter of the NeMo oracle venv, for canary/parakeet baselines. |
|
path |
unset |
Python interpreter of the ONNX oracle venv, for parakeet baselines. |
|
path |
unset |
Python interpreter of the MLX reference venv (mlx-audio, mlx-lm), for the Apple-GPU baselines. |
|
path |
unset |
Python interpreter of the TTS reference venv (kokoro, kittentts, onnxruntime, pocket-tts), for the text-to-speech baselines. |
|
path |
unset |
Python interpreter of the GigaAM oracle venv (upstream gigaam + torch), for the GigaAM external-reference dumps. |
|
path |
unset |
Python interpreter of the translation oracle venv (numpy, sentencepiece), for the Bergamot marian stage dumps. |
|
text |
set by Windows |
Read to detect Windows (Windows_NT); set by the OS, not by dasLLAMA. |
|
text |
set by Windows |
Read for the CPU architecture on Windows; set by the OS, not by dasLLAMA. |
|
number |
set by Windows |
Read for the core count on Windows; set by the OS, not by dasLLAMA. |
11.10. Tests
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
flag |
off |
Run the FULL parity matrix. Without it the big-model rows silently skip. |
|
text |
unset |
Comma-separated filter restricting which parity arms run. |
|
text |
unset |
Comma-separated filter restricting which model families run. |
|
path |
unset |
Directory of llama2.c reference checkpoints for the forward/decode parity tests. |
|
path |
unset |
Directory of whisper models for the audio tests. |
|
text |
unset |
Ambient platform variable; read only by the browser examples’ smoke tests (storyteller, storywish, parrot) to tell whether a Linux box has a window server for the example’s window. |
|
text |
unset |
Ambient platform variable (the Wayland twin of DISPLAY); read only by the browser examples’ smoke tests. |
|
path |
unset |
Directory of audio corpus files for the transcription tests. |
|
path |
/tmp |
Scratch directory for test artifacts; set by the OS on macOS. |
|
path |
unset |
Windows scratch-directory fallback when TMPDIR is unset (the test runner’s log dir). |
11.11. Examples and tutorials
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
path |
unset |
Model path for the dasLLAMA tutorials when none is passed on the command line. |
11.12. Sidecar exchange
Overrides for the dasllama.io exchange client.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
text |
unset |
Sidecar exchange base URL override (tests, mirrors); unset = the baked-in dasllama.io. Setting it is an expressed choice to use the exchange - the first-contact consent question is skipped (announced at boot). |
|
text |
unset |
One-shot exchange accept-policy override: verified |
11.13. daslang core knobs dasLLAMA honours
Owned by daslang, not by dasLLAMA - listed because dasLLAMA’s behaviour depends on them. DAS_TUNE_* is covered in skills/tune.md.
Variable |
Type |
Default |
Effect |
|---|---|---|---|
|
path |
~/.cache/huggingface |
Hugging Face hub root; fetch_models –convert resolves checkpoint blobs under <HF_HOME>/hub. Falls back to HOME’s default cache location when unset. |
|
path |
unset |
Ambient platform variable; read to derive the per-user ~/.dasllama directory (when USERPROFILE is unset) and the default Hugging Face cache when HF_HOME is unset. |
|
path |
unset |
Ambient platform variable (Windows twin of HOME); read only to derive the per-user ~/.dasllama directory. |
|
number |
conservative default |
Total compute lanes for job queues (N-1 workers plus the caller). Overrides set_jobque_threads_cap; see skills/internal/environment_variables.md. |
|
number |
2 on darwin (engine [init]), else 0 |
Worker affinity: 0 off, 1 ideal-processor hint, 2 hard mask - on darwin mode 2 is QoS classification (no pin API) and dasLLAMA defaults it on. Matters on big SMT boxes. |
|
number |
profile-driven |
Team-dispatch rank gate. When set, it suppresses the box profile’s own team_rank_gate knob. |
|
path |
Kernel-tuning sidecar to read/write. Point it somewhere writable when the app dir is read-only. |
|
|
text |
unset |
Kernel-tuning mode. The [tune] framework owns these; see skills/tune.md. |
|
text |
warning (daslang’s to_log floor); tests/run.das sets info for itself and its children when unset |
The to_log floor daslang reads per call (trace, debug, info, warning, error or critical); see skills/internal/environment_variables.md. |
|
text |
DAS_LOG_LEVEL, then warning |
The daslib/logger floor, read once at init; it wins over DAS_LOG_LEVEL where both are set. See skills/internal/environment_variables.md. |
|
text |
unset |
Kernel-tuning policy override. The [tune] framework owns these; see skills/tune.md. |
|
text |
unset |
Semicolon-separated kernel fallbacks for the generated tune probe. |
|
flag |
off |
Compile the jobque marker rail in (read at COMPILE time by daslib/build_const). Without it trace_tag/trace_marker erase from the program; decode_prof –trace refuses on a build without it. |