17.2. dasLLAMA text to speech: models, voices, synthesis

Text to speech in pure daslang: load a converted StyleTTS2-lineage GGUF (KittenTTS nano and mini, Kokoro-82M), run text through the das-native front end (normalizer, part-of-speech tagger, grapheme-to-phoneme), and synthesize mono f32 PCM per sentence chunk, timed per model stage. Run with -jit; utils/dasllama-server/txt2wav.das is the canonical program shape, and the server’s /v1/audio/speech route serves the same facade.

The front-end packs tts_g2p.bin and tts_postag.bin load from the GGUF’s directory. A model’s voices and sample rate come from caps; the rows GEMMs serve as Q8_0 by default, and set_tts_q8 pins the lane for the loads that follow.

Hands-on: the text to speech tutorial.

17.2.1. Constants

TTS_CHUNK_CHARS = 400

TTS_CHUNK_CHARS:int const

17.2.2. Enumerations

TtsKind

enum TtsKind

17.2.3. Structures

TtsModel

A loaded TTS model: the shared assembly, the family’s driver data, and the front-end packs read beside the GGUF. Load with load_tts_model; delete frees every plane and pack.

Fields:

  • kind : TtsKind - the family the GGUF’s architecture named

  • model : StyleTts2Model = StyleTts2Model() - the shared StyleTTS2-lineage assembly, its voices and the family’s driver data

  • scratch : St2Scratch = St2Scratch() - the activation carrier every synthesis reuses

  • g2p : G2pModel - tts_g2p.bin: lexicons, rules and the fallback chain

  • tagger : PosTagger - tts_postag.bin: the tokenizer and part-of-speech tagger

  • pocket : PocketModel = PocketModel() - the Pocket TTS family’s model, when the GGUF is one; it reads text, not phonemes

  • pocket_scratch : PocketScratch = struct<dasllama_pocket::PocketScratch>(uninitialized ) - its activation carrier

  • chunk_chars : int = TTS_CHUNK_CHARS - the longest chunk a synthesis speaks, in codepoints (tts_set_chunk_chars)

TtsVoicePrompt

Portable Pocket codec latents. Prepare with an encoder-bearing model; register with a compatible model, including a compact file without the encoder. Serialization is caller-owned.

Fields:

  • version : int = 1 - Serialized prompt format version.

  • family : string = “pocket” - Model family tag.

  • language : string - Language expected by the receiving model.

  • sample_rate : int - Reference audio sample rate in hertz.

  • latent_dim : int64 - Number of codec latent values per frame.

  • frame_samples : int64 - Number of PCM samples represented by each frame.

  • latents : array<float> - Flattened codec latent frames.

TtsCaps

What a loaded TTS model can do: its voices (canonical names; aliases resolve in the family file), the PCM rate it emits, the languages it speaks, whether it clones a voice from audio, and whether a speed means anything to it.

Fields:

  • voices : array<string> - canonical voice names, in the model’s order

  • voice_langs : array<string> - the language each voice speaks, by index into voices

  • sample_rate : int - PCM samples per second the model emits

  • langs : array<string> - the languages it speaks, as codes

  • cloning : bool - whether a voice can be cloned from audio

  • speed : bool - whether a speed scales the durations; a model without one refuses any speed but 1.0

TtsTimings

Where a synthesis spent its time, in microseconds of wall clock, model loading excluded: the text front end, then each model stage. total_us covers front end through waveform.

Fields:

  • front_end_us : int64 - normalize, tag, phonemize, map into the family’s symbols

  • bert_us : int64 - the PL-BERT pass

  • text_enc_us : int64 - the text encoder

  • duration_us : int64 - the duration encoder and the alignment

  • prosody_us : int64 - F0 and energy

  • decoder_us : int64 - the decoder

  • source_us : int64 - the harmonic-plus-noise source

  • generator_us : int64 - the iSTFTNet generator

  • prompt_us : int64 - the voice and text prompt through the backbone (the continuous-audio families)

  • backbone_us : int64 - the frame loop’s backbone steps

  • head_us : int64 - the frame loop’s flow head

  • codec_us : int64 - latents to PCM through the codec decoder

  • total_us : int64 - front end through waveform

  • samples : int64 - PCM samples produced

TtsAudio

Synthesized speech: mono f32 PCM at sample_rate, with the time it took.

Fields:

  • pcm : array<float> - mono samples in [-1, 1]

  • sample_rate : int - samples per second

  • timings : TtsTimings - where the synthesis spent its time

TtsNoise

The source noise a synthesis consumed - captured from the oracle for a parity run, or drawn from the session’s own generator into a carrier every synthesis reuses.

Fields:

  • uniform : array<float> - one initial phase per harmonic; drawn per synthesis into a reused carrier

  • normal : array<float> - [samples x harmonics], sample-major

  • captured : bool - supplied by an oracle dump: a synthesis reads it as is and never redraws

KittenFamily

struct KittenFamily

KokoroFamily

struct KokoroFamily

17.2.4. Loading and capabilities

caps(m: TtsModel ): TtsCaps

What the loaded model offers: the canonical voice names the front end can drive, each with its language (a voice whose language it does not phonemize is left out; aliases resolve at synthesis), the PCM rate, the languages, whether it clones a voice, whether a speed applies.

Arguments:
finalize(m: TtsModel )

Free the model’s planes, the carrier and both packs; delete m runs it.

Arguments:
g2p_pack_path(dir: string ): string

def g2p_pack_path (dir: string) : string

Arguments:

  • dir : string

load_tts_model(path: string ): TtsModel

Load a TTS model: a converted GGUF (KittenTTS, Kokoro or Pocket TTS, picked from its architecture) or its prepared .dlim image; the phoneme families read tts_g2p.bin and tts_postag.bin from the same directory (Pocket needs neither). A GGUF serves the rows GEMMs on the lane tts_serves_q8 names; an image serves the lane it was baked on.

Arguments:

  • path : string

tts_has_phonemes(m: TtsModel ): bool

Whether tts_phonemize has an answer for this model: the phoneme families yes, a Pocket model no - it reads text, and its chunks carry no phoneme string.

Arguments:
tts_needs_packs(path: string ): bool

Whether the model at path reads the front-end packs (tts_g2p.bin, tts_postag.bin) from its directory: the phoneme families do, a Pocket TTS GGUF reads text and needs neither - decided before any load, by the architecture test load_tts_model makes (a .dlim is a phoneme family).

Arguments:

  • path : string

tts_voice_lang(m: TtsModel; voice: string ): string

The language the front end reads voice in - a canonical name from caps or a family alias, resolved the way synthesize resolves it. A voice this model cannot drive panics here with the message a synthesis would give, rather than making the caller speak a line.

Arguments:

17.2.5. Voices

tts_prepare_voice_prompt(m: TtsModel; pcm: array<float>; sample_rate: int ): TtsVoicePrompt

Encode a reference once for shipping as a voice prompt. The original model revision’s language/codec must match the receiving model; incompatible data is rejected on registration.

Arguments:

  • m : TtsModel

  • pcm : array<float>

  • sample_rate : int

tts_register_voice(m: TtsModel; name: string; pcm: array<float>; sample_rate: int )

Clone a voice: pcm (mono, at the model’s own sample rate - caps().sample_rate) joins the roster under name and speaks from the next synthesis on. A model whose caps() says it cannot clone panics here.

Arguments:

  • m : TtsModel

  • name : string

  • pcm : array<float>

  • sample_rate : int

tts_register_voice_prompt(m: TtsModel; name: string; prompt: TtsVoicePrompt ): bool

Add or replace a named prepared voice without a codec encoder. Failure preserves the roster; the prompt is copied and its backbone state is built lazily at the first synthesis.

Arguments:
tts_voice_prompt_valid(m: TtsModel; prompt: TtsVoicePrompt ): bool

Validate bounded finite latent frames and the model’s language/codec geometry before registration.

Arguments:

17.2.6. Text front end

17.2.6.1. tts_chunks

tts_chunks(m: TtsModel; norm: string ): array<string>

The chunks a synthesis on this model takes, by the family’s own driver rule: Kitten’s appends a comma to a chunk the split left bare, Kokoro’s sends the text as it is (its voices render an added mark as a breath), Pocket’s splits on its tokenizer’s sentence marks under a token budget.

Arguments:
tts_chunks(norm: string; max_len: int = TTS_CHUNK_CHARS; ensure_punct: bool = true ): array<string>
tts_normalize(text: string ): string

The spoken form of text: the normalization pass every synthesis runs first (rules alone - it reads no pack), where “Dr.” reads “Doctor”, “3.5” reads “three point five” and “$12” reads “twelve dollars”.

Arguments:

  • text : string

17.2.6.2. tts_phonemize

tts_phonemize(m: TtsModel; text: string ): string

The phonemes one already-normalized sentence becomes, in the front end’s own American inventory (the tagger of tts_postag.bin labels the tokens, the grapheme-to-phoneme pass of tts_g2p.bin reads them); a family wanting other symbols rewrites them. A Pocket model panics.

Arguments:
tts_phonemize(m: TtsModel; text: string; lang: string ): string

17.2.7. Synthesis

synthesize(m: TtsModel; text: string; voice: string; speed: float = 1f ): TtsAudio

Text -> speech: normalize, chunk by sentence, phonemize, map into the family’s symbols, synthesize; the chunks concatenate, the timings sum. Sample-identical to synthesize_stream over the same text.

Arguments:

  • m : TtsModel

  • text : string

  • voice : string

  • speed : float

synthesize_stream(m: TtsModel; text: string; voice: string; speed: float; blk: block<(chunk:TtsAudio):auto> ): auto

Text -> speech, one sentence-sized chunk at a time: the block receives each chunk’s audio as soon as it exists, in order. Normalization runs once over the whole text; voice is a name from caps or a family alias, and an unknown one panics.

Arguments:

  • m : TtsModel

  • text : string

  • voice : string

  • speed : float

  • blk : block<(chunk: TtsAudio):auto>

tts_release_scratch(m: TtsModel )

Free the activation scratch a synthesis or a clone left behind - the carrier of either family (a StyleTTS2 carrier whose noise the parity rail captured stays) and the block home’s scratch globals; the next run grows them back for a few milliseconds. For a platform where idle memory matters, called when nothing is queued.

Arguments:
tts_set_chunk_chars(m: TtsModel; chars: int )

Cap the chunk a synthesis speaks at chars codepoints (default TTS_CHUNK_CHARS; under one is refused): the peak memory a say holds is the largest chunk’s, so a memory-tight platform lowers it and the same text speaks in more, shorter pieces, each its own synthesis. Pocket’s chunks take no cap.

Arguments:

17.2.8. The served lane

reset_tts_q8()

Drop the set_tts_q8 pin: the next load follows the policy default again.

set_tts_q8(on: bool )

Pin the GEMM weights’ format for subsequent TTS loads of every family: Q8_0 quants (their own prepared image beside the GGUF) or the file’s f32 planes; reset_tts_q8 returns to the policy default. The pin is per context: a worker thread pins for itself.

Arguments:

  • on : bool

tts_serves_q8(): bool

Would the next TTS load serve its rows GEMMs as q8 - the pin when set, the policy otherwise.

17.2.9. Timings

rtf(t: TtsTimings; sample_rate: int ): double

Real-time factor: generation seconds per second of audio produced.

Arguments:
timings_line(t: TtsTimings; sample_rate: int ): string

One line for the log: seconds of audio, wall time, RTF and the per-stage split in ms - the stages a family ran.

Arguments: