17.2. dasLLAMA text to speech: models, voices, synthesis

Text to speech in pure daslang: load a converted StyleTTS2-lineage GGUF (KittenTTS nano and mini, Kokoro-82M), run text through the das-native front end (normalizer, part-of-speech tagger, grapheme-to-phoneme), and synthesize mono f32 PCM per sentence chunk, timed per model stage. Run with -jit; utils/dasllama-server/txt2wav.das is the canonical program shape, and the server’s /v1/audio/speech route serves the same facade.

The front-end packs tts_g2p.bin and tts_postag.bin load from the GGUF’s directory. A model’s voices and sample rate come from caps; the rows GEMMs serve as Q8_0 by default, and set_tts_q8 pins the lane for the loads that follow.

Hands-on: the text to speech tutorial.

17.2.1. Constants

TTS_CHUNK_CHARS = 400

TTS_CHUNK_CHARS:int const

17.2.2. Enumerations

TtsKind

enum TtsKind

17.2.3. Structures

TtsModel

A loaded TTS model: the shared assembly, the family’s driver data, and the front-end packs read beside the GGUF. Load with load_tts_model; delete frees every plane and pack.

Fields:

  • kind : TtsKind - the family the GGUF’s architecture named

  • model : StyleTts2Model = StyleTts2Model() - the shared StyleTTS2-lineage assembly and its voices

  • scratch : St2Scratch = St2Scratch() - the activation carrier every synthesis reuses

  • kitten : KittenFamily - the KittenTTS driver data (symbol table, aliases, speed priors)

  • kokoro : KokoroFamily - the Kokoro driver data (vocabulary)

  • g2p : G2pModel - tts_g2p.bin: lexicons, rules and the fallback chain

  • tagger : PosTagger - tts_postag.bin: the tokenizer and part-of-speech tagger

TtsCaps

What a loaded TTS model can do: its voices (canonical names; aliases resolve in the family file), the PCM rate it emits, the languages it speaks, whether it clones a voice from audio (none of the first families do).

Fields:

  • voices : array<string> - canonical voice names, in the model’s order

  • voice_langs : array<string> - the language each voice speaks, by index into voices

  • sample_rate : int - PCM samples per second the model emits

  • langs : array<string> - the languages it speaks, as codes

  • cloning : bool - whether a voice can be cloned from audio

TtsTimings

Where a synthesis spent its time, in microseconds of wall clock, model loading excluded: the text front end, then each model stage. total_us covers front end through waveform.

Fields:

  • front_end_us : int64 - normalize, tag, phonemize, map into the family’s symbols

  • bert_us : int64 - the PL-BERT pass

  • text_enc_us : int64 - the text encoder

  • duration_us : int64 - the duration encoder and the alignment

  • prosody_us : int64 - F0 and energy

  • decoder_us : int64 - the decoder

  • source_us : int64 - the harmonic-plus-noise source

  • generator_us : int64 - the iSTFTNet generator

  • total_us : int64 - front end through waveform

  • samples : int64 - PCM samples produced

TtsAudio

Synthesized speech: mono f32 PCM at sample_rate, with the time it took.

Fields:

  • pcm : array<float> - mono samples in [-1, 1]

  • sample_rate : int - samples per second

  • timings : TtsTimings - where the synthesis spent its time

TtsNoise

The source noise a synthesis consumed - captured from the oracle for a parity run, or drawn from the session’s own generator into a carrier every synthesis reuses.

Fields:

  • uniform : array<float> - one initial phase per harmonic; drawn per synthesis into a reused carrier

  • normal : array<float> - [samples x harmonics], sample-major

  • captured : bool - supplied by an oracle dump: a synthesis reads it as is and never redraws

17.2.4. Loading and capabilities

caps(m: TtsModel ): TtsCaps

What the loaded model offers: the canonical voice names the front end can drive, each with its language (a voice whose language it does not phonemize is left out; a family’s aliases resolve at synthesis), the PCM sample rate, the languages, and whether it clones a voice.

Arguments:
finalize(m: TtsModel )

Free the model’s planes, the carrier and both packs; delete m runs it.

Arguments:
load_tts_model(path: string ): TtsModel

Load a TTS GGUF (KittenTTS or Kokoro, picked from its architecture); tts_g2p.bin and tts_postag.bin come from the same directory. The rows GEMMs serve as the lane tts_serves_q8 names at the time of the load.

Arguments:

  • path : string

tts_voice_lang(m: TtsModel; voice: string ): string

The language the front end reads voice in - a canonical name from caps or a family alias, resolved the way synthesize resolves it. A voice this model cannot drive panics here with the message a synthesis would give, rather than making the caller speak a line.

Arguments:

17.2.5. Text front end

17.2.5.1. tts_chunks

tts_chunks(m: TtsModel; norm: string ): array<string>

The chunks a synthesis on this model takes: the family’s own driver rule - Kitten’s appends a comma to a chunk the split left bare, Kokoro’s pipeline sends the text as it is, and its voices render an added mark as an audible breath.

Arguments:
tts_chunks(norm: string; max_len: int = TTS_CHUNK_CHARS; ensure_punct: bool = true ): array<string>
tts_normalize(text: string ): string

The spoken form of text: the normalization pass every synthesis runs first (rules alone - it reads no pack), where “Dr.” reads “Doctor”, “3.5” reads “three point five” and “$12” reads “twelve dollars”.

Arguments:

  • text : string

17.2.5.2. tts_phonemize

tts_phonemize(m: TtsModel; text: string ): string

The phonemes one already-normalized sentence becomes, in the front end’s own American inventory: the tagger of tts_postag.bin labels the tokens, the grapheme-to-phoneme pass of tts_g2p.bin reads them. A family wanting other symbols rewrites this string into them.

Arguments:
tts_phonemize(m: TtsModel; text: string; lang: string ): string

17.2.6. Synthesis

synthesize(m: TtsModel; text: string; voice: string; speed: float = 1f ): TtsAudio

Text -> speech: normalize, chunk by sentence, phonemize, map into the family’s symbols, synthesize; the chunks concatenate, the timings sum. Sample-identical to synthesize_stream over the same text.

Arguments:

  • m : TtsModel

  • text : string

  • voice : string

  • speed : float

synthesize_stream(m: TtsModel; text: string; voice: string; speed: float; blk: block<(chunk:TtsAudio):auto> ): auto

Text -> speech, one sentence-sized chunk at a time: the block receives each chunk’s audio as soon as it exists, in order. Normalization runs once over the whole text; voice is a name from caps or a family alias, and an unknown one panics.

Arguments:

  • m : TtsModel

  • text : string

  • voice : string

  • speed : float

  • blk : block<(chunk: TtsAudio):auto>

17.2.7. The served lane

reset_tts_q8()

Drop the set_tts_q8 pin: the next load follows the policy default again.

set_tts_q8(on: bool )

Pin the rows GEMMs’ format for subsequent TTS loads: Q8_0 quants (their own prepared image beside the GGUF) or the file’s f32 planes; reset_tts_q8 returns to the policy default. The pin is per context: a worker thread pins for itself.

Arguments:

  • on : bool

tts_serves_q8(): bool

Would the next TTS load serve its rows GEMMs as q8 - the pin when set, the policy otherwise.

17.2.8. Timings

rtf(t: TtsTimings; sample_rate: int ): double

Real-time factor: generation seconds per second of audio produced.

Arguments:
timings_line(t: TtsTimings; sample_rate: int ): string

One line for the log: seconds of audio, wall time, RTF and the per-stage split in ms.

Arguments: