dasLLAMA is an LLM + speech-to-text stack written entirely in daslang and JIT-compiled — no hand-written assembly. These are CPU-only comparisons. llama.cpp is a broader, more mature engine with GPU backends we don't try to match; on the CPU path alone, das keeps pace on prompt processing and often edges ahead.
// LLM inference in daslang — JIT, no C++
require dasllama/dasllama
require daslib/jobque_boost
[export]
def main() {
let path = "Llama-3.1-8B.gguf"
var m <- load_model(path, QuantMode.q8)
let ids <- encode(m, "Summarize the results:")
with_job_que() {
setup_dasllama_jobque()
var s = create_session(m)
let sp = SamplingParams()
generate(m, s, ids, sp, 128l) $(_id, p) {
print(p)
return true
}
}
}A like-for-like CPU comparison: llama.cpp built with no Accelerate/AMX BLAS, so this reflects scalar/SIMD kernels — not its GPU backends, which are out of scope here. pp512 = prefill over a 512-token prompt; tg128 = greedy generation of 128 tokens. All Q8.
| model | active B | pp512 das | pp512 lcpp | das/lcpp | tg128 das | tg128 lcpp | das/lcpp |
|---|
Transcription time, CPU only, greedy decode. Bars show speed relative to the reference CLI (longer = faster); the small figure is real-time factor — seconds of audio per second of compute. Note the macOS *-cli tools do get Apple's AMX matrix unit here, so this isn't scalar-only on their side.
This is a narrow, CPU-only slice: llama.cpp's Conformer/AuT audio encoders don't route through BLAS/AMX on CPU, so the reference spends ~99% of its time in the audio encoder. das's optimized fp32 scalar encoder comes out well ahead on that path — a GPU build of llama.cpp would tell a different story.
Canary-Qwen 2.5B (the only engine with a SALM reference, NeMo) — das leads on short dictation clips: 0.66× on jfk, 0.77× on jfk3. It trails on a 3-minute clip where it runs an un-quantized fp32 decoder — a known follow-up.