.. _stdlib_dasllama_tts: ================================================== dasLLAMA text to speech: models, voices, synthesis ================================================== .. das:module:: dasllama_tts Text to speech in pure daslang: load a converted StyleTTS2-lineage GGUF (KittenTTS nano and mini, Kokoro-82M), run text through the das-native front end (normalizer, part-of-speech tagger, grapheme-to-phoneme), and synthesize mono f32 PCM per sentence chunk, timed per model stage. Run with ``-jit``; ``utils/dasllama-server/txt2wav.das`` is the canonical program shape, and the server's ``/v1/audio/speech`` route serves the same facade. The front-end packs ``tts_g2p.bin`` and ``tts_postag.bin`` load from the GGUF's directory. A model's voices and sample rate come from ``caps``; the rows GEMMs serve as Q8_0 by default, and ``set_tts_q8`` pins the lane for the loads that follow. Hands-on: :ref:`the text to speech tutorial `. +++++++++ Constants +++++++++ .. _global-dasllama_tts-TTS_CHUNK_CHARS: .. das:attribute:: TTS_CHUNK_CHARS = 400 TTS_CHUNK_CHARS:int const ++++++++++++ Enumerations ++++++++++++ .. _enum-dasllama_tts-TtsKind: .. das:attribute:: TtsKind enum TtsKind ++++++++++ Structures ++++++++++ .. _struct-dasllama_tts-TtsModel: .. das:attribute:: TtsModel A loaded TTS model: the shared assembly, the family's driver data, and the front-end packs read beside the GGUF. Load with ``load_tts_model``; ``delete`` frees every plane and pack. :Fields: * **kind** : :ref:`TtsKind ` - the family the GGUF's architecture named * **model** : StyleTts2Model = StyleTts2Model() - the shared StyleTTS2-lineage assembly and its voices * **scratch** : St2Scratch = St2Scratch() - the activation carrier every synthesis reuses * **kitten** : KittenFamily - the KittenTTS driver data (symbol table, aliases, speed priors) * **kokoro** : KokoroFamily - the Kokoro driver data (vocabulary) * **g2p** : G2pModel - ``tts_g2p.bin``: lexicons, rules and the fallback chain * **tagger** : PosTagger - ``tts_postag.bin``: the tokenizer and part-of-speech tagger .. _struct-dasllama_tts_types-TtsCaps: .. das:attribute:: TtsCaps What a loaded TTS model can do: its voices (canonical names; aliases resolve in the family file), the PCM rate it emits, the languages it speaks, whether it clones a voice from audio (none of the first families do). :Fields: * **voices** : array - canonical voice names, in the model's order * **voice_langs** : array - the language each voice speaks, by index into ``voices`` * **sample_rate** : int - PCM samples per second the model emits * **langs** : array - the languages it speaks, as codes * **cloning** : bool - whether a voice can be cloned from audio .. _struct-dasllama_tts_types-TtsTimings: .. das:attribute:: TtsTimings Where a synthesis spent its time, in microseconds of wall clock, model loading excluded: the text front end, then each model stage. ``total_us`` covers front end through waveform. :Fields: * **front_end_us** : int64 - normalize, tag, phonemize, map into the family's symbols * **bert_us** : int64 - the PL-BERT pass * **text_enc_us** : int64 - the text encoder * **duration_us** : int64 - the duration encoder and the alignment * **prosody_us** : int64 - F0 and energy * **decoder_us** : int64 - the decoder * **source_us** : int64 - the harmonic-plus-noise source * **generator_us** : int64 - the iSTFTNet generator * **total_us** : int64 - front end through waveform * **samples** : int64 - PCM samples produced .. _struct-dasllama_tts_types-TtsAudio: .. das:attribute:: TtsAudio Synthesized speech: mono f32 PCM at ``sample_rate``, with the time it took. :Fields: * **pcm** : array - mono samples in [-1, 1] * **sample_rate** : int - samples per second * **timings** : :ref:`TtsTimings ` - where the synthesis spent its time .. _struct-dasllama_tts_types-TtsNoise: .. das:attribute:: TtsNoise The source noise a synthesis consumed - captured from the oracle for a parity run, or drawn from the session's own generator into a carrier every synthesis reuses. :Fields: * **uniform** : array - one initial phase per harmonic; drawn per synthesis into a reused carrier * **normal** : array - [samples x harmonics], sample-major * **captured** : bool - supplied by an oracle dump: a synthesis reads it as is and never redraws ++++++++++++++++++++++++ Loading and capabilities ++++++++++++++++++++++++ * :ref:`caps (m: TtsModel) : TtsCaps ` * :ref:`finalize (var m: TtsModel) ` * :ref:`load_tts_model (path: string) : TtsModel ` * :ref:`tts_voice_lang (m: TtsModel; voice: string) : string ` .. _function-dasllama_tts_caps_TtsModel: .. das:function:: caps(m: TtsModel) : TtsCaps What the loaded model offers: the canonical voice names the front end can drive, each with its language (a voice whose language it does not phonemize is left out; a family's aliases resolve at synthesis), the PCM sample rate, the languages, and whether it clones a voice. :Arguments: * **m** : :ref:`TtsModel ` .. _function-dasllama_tts_finalize_TtsModel: .. das:function:: finalize(m: TtsModel) Free the model's planes, the carrier and both packs; ``delete m`` runs it. :Arguments: * **m** : :ref:`TtsModel ` .. _function-dasllama_tts_load_tts_model_string: .. das:function:: load_tts_model(path: string) : TtsModel Load a TTS GGUF (KittenTTS or Kokoro, picked from its architecture); ``tts_g2p.bin`` and ``tts_postag.bin`` come from the same directory. The rows GEMMs serve as the lane ``tts_serves_q8`` names at the time of the load. :Arguments: * **path** : string .. _function-dasllama_tts_tts_voice_lang_TtsModel_string: .. das:function:: tts_voice_lang(m: TtsModel; voice: string) : string The language the front end reads ``voice`` in - a canonical name from ``caps`` or a family alias, resolved the way ``synthesize`` resolves it. A voice this model cannot drive panics here with the message a synthesis would give, rather than making the caller speak a line. :Arguments: * **m** : :ref:`TtsModel ` * **voice** : string ++++++++++++++ Text front end ++++++++++++++ * :ref:`tts_chunks (m: TtsModel; norm: string) : array\

` * :ref:`tts_chunks (norm: string; max_len: int = TTS_CHUNK_CHARS; ensure_punct: bool = true) : array\

` * :ref:`tts_normalize (text: string) : string ` * :ref:`tts_phonemize (m: TtsModel; text: string) : string ` * :ref:`tts_phonemize (m: TtsModel; text: string; lang: string) : string ` tts_chunks ^^^^^^^^^^ .. _function-dasllama_tts_tts_chunks_TtsModel_string: .. das:function:: tts_chunks(m: TtsModel; norm: string) : array The chunks a synthesis on this model takes: the family's own driver rule - Kitten's appends a comma to a chunk the split left bare, Kokoro's pipeline sends the text as it is, and its voices render an added mark as an audible breath. :Arguments: * **m** : :ref:`TtsModel ` * **norm** : string .. _function-dasllama_tts_tts_chunks_string_int_bool: .. das:function:: tts_chunks(norm: string; max_len: int = TTS_CHUNK_CHARS; ensure_punct: bool = true) : array ---- .. _function-dasllama_tts_tts_normalize_string: .. das:function:: tts_normalize(text: string) : string The spoken form of ``text``: the normalization pass every synthesis runs first (rules alone - it reads no pack), where "Dr." reads "Doctor", "3.5" reads "three point five" and "$12" reads "twelve dollars". :Arguments: * **text** : string tts_phonemize ^^^^^^^^^^^^^ .. _function-dasllama_tts_tts_phonemize_TtsModel_string: .. das:function:: tts_phonemize(m: TtsModel; text: string) : string The phonemes one already-normalized sentence becomes, in the front end's own American inventory: the tagger of ``tts_postag.bin`` labels the tokens, the grapheme-to-phoneme pass of ``tts_g2p.bin`` reads them. A family wanting other symbols rewrites this string into them. :Arguments: * **m** : :ref:`TtsModel ` * **text** : string .. _function-dasllama_tts_tts_phonemize_TtsModel_string_string: .. das:function:: tts_phonemize(m: TtsModel; text: string; lang: string) : string +++++++++ Synthesis +++++++++ * :ref:`synthesize (var m: TtsModel; text: string; voice: string; speed: float = 1f) : TtsAudio ` * :ref:`synthesize_stream (var m: TtsModel; text: string; voice: string; speed: float; blk: block\<(chunk:TtsAudio):auto\>) : auto ` .. _function-dasllama_tts_synthesize_TtsModel_string_string_float: .. das:function:: synthesize(m: TtsModel; text: string; voice: string; speed: float = 1f) : TtsAudio Text -> speech: normalize, chunk by sentence, phonemize, map into the family's symbols, synthesize; the chunks concatenate, the timings sum. Sample-identical to ``synthesize_stream`` over the same text. :Arguments: * **m** : :ref:`TtsModel ` * **text** : string * **voice** : string * **speed** : float .. _function-dasllama_tts_synthesize_stream_TtsModel_string_string_float_block_ls_chunk_c_TtsAudio_c_auto_gr_: .. das:function:: synthesize_stream(m: TtsModel; text: string; voice: string; speed: float; blk: block<(chunk:TtsAudio):auto>) : auto Text -> speech, one sentence-sized chunk at a time: the block receives each chunk's audio as soon as it exists, in order. Normalization runs once over the whole text; ``voice`` is a name from ``caps`` or a family alias, and an unknown one panics. :Arguments: * **m** : :ref:`TtsModel ` * **text** : string * **voice** : string * **speed** : float * **blk** : block<(chunk: :ref:`TtsAudio `):auto> +++++++++++++++ The served lane +++++++++++++++ * :ref:`reset_tts_q8 () ` * :ref:`set_tts_q8 (on: bool) ` * :ref:`tts_serves_q8 () : bool ` .. _function-dasllama_tts_reset_tts_q8: .. das:function:: reset_tts_q8() Drop the ``set_tts_q8`` pin: the next load follows the policy default again. .. _function-dasllama_tts_set_tts_q8_bool: .. das:function:: set_tts_q8(on: bool) Pin the rows GEMMs' format for subsequent TTS loads: Q8_0 quants (their own prepared image beside the GGUF) or the file's f32 planes; ``reset_tts_q8`` returns to the policy default. The pin is per context: a worker thread pins for itself. :Arguments: * **on** : bool .. _function-dasllama_tts_tts_serves_q8: .. das:function:: tts_serves_q8() : bool Would the next TTS load serve its rows GEMMs as q8 - the pin when set, the policy otherwise. +++++++ Timings +++++++ * :ref:`rtf (t: TtsTimings; sample_rate: int) : double ` * :ref:`timings_line (t: TtsTimings; sample_rate: int) : string ` .. _function-dasllama_tts_types_rtf_TtsTimings_int: .. das:function:: rtf(t: TtsTimings; sample_rate: int) : double Real-time factor: generation seconds per second of audio produced. :Arguments: * **t** : :ref:`TtsTimings ` * **sample_rate** : int .. _function-dasllama_tts_types_timings_line_TtsTimings_int: .. das:function:: timings_line(t: TtsTimings; sample_rate: int) : string One line for the log: seconds of audio, wall time, RTF and the per-stage split in ms. :Arguments: * **t** : :ref:`TtsTimings ` * **sample_rate** : int