loop-tts — self-verifying TTS for agents and pipelines

Your TTS engine does not tell you what it actually said. loop-tts closes the loop: every utterance is generated, transcribed by an independent ASR (Whisper), measured, and re-generated with escalating instructions until it passes quality gates. The result is a WAV file and a structured QC report.

              ┌──────────────────────────────────────────────────────┐
 text ──────▶ │  TTS backend (any HTTP endpoint: POST text → WAV)    │
              └──────────────────────────┬───────────────────────────┘
                                         ▼
              ┌──────────────────────────────────────────────────────┐
              │  independent ASR (faster-whisper) transcribes it     │
              └──────────────────────────┬───────────────────────────┘
                                         ▼
              ┌──────────────────────────────────────────────────────┐
              │  quality gates on what was ACTUALLY said:            │
              │   • real WPM   ≥ target (words transcribed / seconds)│
              │   • integrity  ≥ min   (no swallowed words)          │
              │   • max gap    ≤ limit (no mid-sentence dead air)    │
              └──────────────────────────┬───────────────────────────┘
                                       pass?  no ──▶ escalate (see below)
                                         │ yes      and retry (keep best)
                                         ▼
                              audio + QCReport

Why

TTS models fail in ways you cannot see by reading the text: they talk too slowly, swallow a word, or freeze for two seconds mid-sentence. The classic "generate → listen → maybe retry" workflow is manual and unreliable. Verifying with an independent ASR makes the quality check objective and automatic — and the QC report is machine-readable, which is exactly what agents need.

Verifying TTS output with ASR has been described recently for audiobook pipelines and in closed-loop TTS research. What loop-tts adds:

1. A complete, dependency-light implementation — one package, any backend.

2. Pluggable escalation (below) instead of a fixed retry strategy.

3. A structured QC report designed for agents — exposed as MCP tools and a REST API, so any agent can read the failure and fix the text itself.

4. verify mode — the quality gates run on audio from any TTS engine, turning the loop into a generic pre-publication QA gate.

Related MCP server: Clerk Chat MCP Server

Escalation strategies

When a chunk fails, the loop retries with a stronger "boost". Which knob it turns depends on your engine's control surface:

The verification loop is engine-agnostic: WPM, integrity, gap detection and keep-best work identically with any backend. Only the escalation ladder is swapped.

Languages

The loop is language-agnostic: it works for any language covered by both your TTS engine and the Whisper verification model (the reference Qwen3-TTS server covers 10 languages; Whisper small covers ~99). Numbers are spelled out with num2words (20+ languages) before comparing the expected text with the transcription. For unspaced scripts (Chinese, Japanese), integrity is computed per character and the rate gate counts characters per minute — set wpm_min accordingly (Mandarin broadcast is roughly 200–260 chars/min).

The default escalation ladder is written in English (it is a style instruction read by the model); translate config.boosts for best results in other languages.

Quality gates

• Real WPM — words in the transcription divided by the real audio duration (not the requested text). Catches slow, "mushy" speech.

• Word integrity — multiset recall of the expected words in the transcription, with numbers spelled out in the target language (1500 ↔ mille cinq cents) so ASR phrasing differences do not false-alarm. Catches swallowed words.

• Max internal gap — longest silence strictly inside the audio (chunk edges are excluded), detected by RMS. Catches mid-sentence dead air.

• Keep best — all attempts are scored (2·ΔWPM + 100·(1−integrity) + 50·Δgap); the best one is kept even if none fully passed.

Install

pip install loop-tts

You need any TTS backend that answers POST /tts with a WAV body (see docs/tts-server.md for a minimal example and notes on open instruction-aware models). The Whisper model (~460 MB for small) is downloaded on first use.

Quickstart (Python)

from loop_tts import TTSLoop, LoopTTSConfig

cfg = LoopTTSConfig(
    tts_url="http://127.0.0.1:8092/tts",   # your TTS backend
    language="fr",                          # fr / en / de / ...
    wpm_min=145,                            # minimum real speaking rate
    escalation="instruct",                  # or "parameter" / "none"
)
loop = TTSLoop(cfg)

path, report = loop.speak(
    "Welcome to the show. Today we talk about sleep, training and one "
    "very opinionated kettlebell.",
    instruct="Warm, conversational tone.",
)
print(report.ok, report.wpm_real, report.integrity, report.attempts)
# True 168.3 1.0 2

CLI

# single utterance (QC report on stdout, exit code 0 = passed)
loop-tts speak "Bonjour, tout le monde." --instruct "Dynamique" --out hello.wav

# multi-voice piece from a JSON file
loop-tts batch segments.json --out episode.wav

# quality-gate ANY existing audio (any TTS engine)
loop-tts verify hello.wav --text "Bonjour, tout le monde."

# HTTP API (localhost by default) — docs at /docs
loop-tts serve --port 8300

# MCP server (stdio)
loop-tts mcp

HTTP API

The server binds to 127.0.0.1 by default. If you expose it, set LOOP_TTS_API_KEY: every route then requires the X-API-Key header.

For agents (MCP)

loop-tts mcp exposes four tools:

• tts_speak(text, speaker?, instruct?, min_wpm?) → audio path + QC report

• tts_batch(segments) → assembled audio + one report per segment

• tts_verify(audio_path, text) → QC report on any existing audio

• tts_status() → health

The interesting part is the report. When report.ok is false, the report tells the agent what failed (failed_checks), what was actually said (transcribed), and per-attempt detail. The agent can then do what a human would do but faster: rewrite the offending text (paraphrase a swallowed proper noun, split a long sentence, remove an ambiguous opening word) and call tts_speak again. The loop escalates the delivery; the agent can escalate the content.

Configuration

All knobs are plain values in LoopTTSConfig (or env vars):

Security

• No secrets in code or config defaults; the TTS URL is a plain setting.

• HTTP API binds to localhost by default; optional LOOP_TTS_API_KEY shared key; audio is only served from the configured out dir (no path traversal).

• Generated audio and QC reports are local files — nothing is uploaded.

Project layout

loop_tts/
├── config.py    # every knob, one place
├── metrics.py   # WPM, word integrity, internal gap detection
├── stt.py       # lazy faster-whisper wrapper
├── core.py      # the loop: chunk → generate → transcribe → measure → escalate
├── api.py       # FastAPI app
├── mcp.py       # MCP server
└── cli.py       # loop-tts speak|batch|verify|serve|mcp

Disclaimer

loop-tts is an independent project. Qwen3-TTS is referenced only as an example of an open instruction-aware TTS model; loop-tts works with any backend exposing a simple HTTP endpoint.

License

MIT — see LICENSE.