low-latency-tts-service-mcp

Low-latency local text-to-speech powered by Kokoro through TTS.cpp. The service keeps the Kokoro GGUF loaded in one long-running local tts-cli process for fast inference, then handles queued playback, status tracking, and MCP integration for AI agents. It offers a FastAPI server for HTTP clients, a TypeScript MCP relay for Claude Code, Claude Desktop, or any MCP-compatible client, and an interactive terminal chat REPL (just chat) for typing text and hearing it spoken right away.

Features

Under the hood, the project converts text to phonemes with misaki, the grapheme-to-phoneme library Kokoro was trained with, then hands them to a local TTS.cpp tts-cli process for Kokoro generation. That process is started once with --serve, loads the Kokoro GGUF a single time, and serves every request and every voice (the GGUF carries all voice packs); it is restarted automatically if it exits. Words missing from misaki's lexicons fall back to the phonemizer built into the Kokoro GGUF, which likewise runs as one persistent phonemize --serve process. It uses sounddevice for audio output, and uses FastAPI for the HTTP server. The MCP server is a lightweight TypeScript stdio-to-HTTP relay using the Model Context Protocol SDK.

Design Principles

All runtime configuration is explicit. If a required value is missing from config.yaml, or if the configured tts-cli executable or Kokoro GGUF model is not present, the service fails immediately with a clear error. The service does not silently fall back to another model, voice, port, binary, or output directory.

Audio files are written to data/output/ as WAV files with timestamps when save_wav: true in config.yaml. The server serializes requests through a queue and a background audio worker. The worker generates audio for a request, starts playback, and can generate the next queued request while the current one is playing.

Related MCP server: MCP TTS Server

Architecture

There are three entry paths into the system. HTTP clients call the FastAPI server directly. AI agents call the MCP relay (mcp/tts-mcp.ts), which reads the server host and port from config.yaml, checks /health, and forwards tool calls to the FastAPI API. Terminal users run the interactive chat REPL (src/main.py), which drives the shared TTS runtime directly without going through the HTTP server. In every case, TTS inference runs out-of-process through the configured TTS.cpp tts-cli, using the Kokoro no-espeak GGUF model.

┌─────────────────────────┐    ┌────────────────────┐      ┌─────────────────────────┐
│        AI Agent         │    │    HTTP Client     │      │      Terminal User      │
│  Claude Code / Desktop  │    │   curl / scripts   │      │        just chat        │
└────────────┬────────────┘    └─────────┬──────────┘      └────────────┬────────────┘
             │ MCP stdio                 │ HTTP                         │ keystrokes
             ▼                           │                              ▼
┌─────────────────────────┐              │                 ┌─────────────────────────┐
│  MCP Server (Node.js)   │              │                 │       src/main.py       │
│     mcp/tts-mcp.ts      │              │                 │  interactive chat REPL  │
│ tools: say, get_voices, │              │                 │ type text -> synth+play │
│       get_status        │              │                 └────────────┬────────────┘
└────────────┬────────────┘              │                              │
             │ HTTP                      │                              │
             ▼                           ▼                              │
┌───────────────────────────────────────────────────────┐               │
│                    FastAPI Server                     │               │
│       src/low_latency_tts_service_mcp/server.py       │               │
│                                                       │               │
│     POST /say  GET /voices  /status/{id}  /health     │               │
│       work queue -> audio worker -> status map        │               │
└───────────────────┬───────────────────────────────────┘               │
                    │                                                   │
                    ▼                                                   ▼
┌────────────────────────────────────────────────────────────────────────────────────┐
│            Shared TTS Runtime - src/low_latency_tts_service_mcp/tts.py             │
│                                                                                    │
│       text cleanup -> TTS.cpp command -> WAV reader -> sounddevice playback        │
└───────────────────┬───────────────────────────────────┬────────────────────────────┘
                    │                                   │
                    ▼                                   ▼
        ┌───────────────────────┐           ┌───────────────────────┐
        │    TTS.cpp tts-cli    │           │   data/output/*.wav   │
        │ Kokoro_no_espeak.gguf │           │   timestamped audio   │
        └───────────┬───────────┘           └───────────────────────┘
                    │
                    ▼
              ┌───────────┐
              │ Speakers  │
              └───────────┘

Prerequisites

• Python 3.12+

• uv - Python package manager (install)

• just - Command runner (install)

• Node.js 18+ - For the MCP server

• git and cmake - Used by just build-tts to fetch, patch and compile TTS.cpp

• Local audio output device - Required for playback through sounddevice

Project Structure

.
├── src/
│   ├── main.py                         # Interactive chat REPL (just chat / just run)
│   ├── server.py                       # Module wrapper for uv run -m src.server
│   └── low_latency_tts_service_mcp/
│       ├── server.py                   # FastAPI server, queue, statuses, worker
│       └── tts.py                      # Config, Kokoro command, WAV playback
├── tests/
│   ├── test_main.py
│   ├── test_server.py
│   ├── test_tts.py
│   ├── test_integration.py
│   └── architecture/                   # Architecture import rule tests
├── scripts/
│   └── download-model.sh               # Interactive Kokoro model downloader
├── mcp/
│   ├── tts-mcp.ts                      # MCP relay to FastAPI server
│   ├── package.json
│   └── tsconfig.json
├── config/
│   ├── semgrep/                        # Static analysis rules
│   └── codespell/                      # Spell-check configuration
├── data/
│   ├── models/                         # Downloaded Kokoro GGUF model
│   └── output/                         # Generated WAV files
├── stubs/                              # Local type stubs for strict checking
├── vendor/
│   └── TTS.cpp/                        # Local TTS.cpp checkout/build location
├── config.yaml                         # Runtime configuration
├── justfile                            # Command recipes
└── pyproject.toml                      # Project metadata and dependencies

Setup

just init

Builds the TTS.cpp binaries, creates report directories, and installs Python dependencies via uv sync --all-extras. If no Kokoro model is present and the command is running interactively, just init prompts for a model download. In non-interactive contexts it tells you to run just download.

Build the TTS.cpp Binaries

just build-tts

Run automatically by just init, and a no-op once the binaries exist — delete vendor/ to force a rebuild. vendor/ is gitignored, so the binaries are a local build artifact; what this repository tracks is everything needed to reproduce them:

The patch adds a --phonemes flag so tts-cli accepts misaki's phoneme string directly instead of re-phonemizing it, keeps ., ! and ? in the phonemized prompt so Kokoro produces sentence-boundary pauses instead of running sentences together, builds the phonemize helper used as the fallback for unknown words, trims unused example targets, adds load and generation timing output, and adds a --serve mode to tts-cli and phonemize that loads the model once and answers one request per stdin line (<voice>\t<save-path>\t<phonemes> for tts-cli, a word for phonemize) with @@done\t<result> or @@error\t<message>. The ggml Accelerate, BLAS and Metal backends are pinned off so every machine builds the same CPU binary and generates identical audio.

To move to a newer upstream TTS.cpp, bump TTS_CPP_COMMIT, delete vendor/, and re-run. If the patch no longer applies, re-create it with git -C vendor/TTS.cpp diff > patches/tts-cpp.patch.

Download a Model

just download

Downloads the Kokoro no-espeak GGUF model used by the default configuration:

The default path is:

data/models/Kokoro_no_espeak.gguf

Getting Started

1. Run just init - builds the TTS.cpp binaries and installs Python dependencies

2. Run just download - downloads Kokoro_no_espeak.gguf if it is not already present

3. Confirm config.yaml points at the local tts-cli executable and downloaded model

4. Run just start - starts the FastAPI TTS server

5. Send requests via HTTP or through the MCP bridge

To try synthesis without the server, run just chat for an interactive terminal REPL: type a line, press Enter twice, and hear it spoken.

Configuration

TTS and server runtime settings live in config.yaml at the project root. Some operational constants, such as status retention and MCP request timeouts, are defined in code. Example:

tts_cli: ./vendor/TTS.cpp/build/bin/tts-cli
phonemize_cli: ./vendor/TTS.cpp/build/bin/phonemize
model: ./data/models/Kokoro_no_espeak.gguf
output_dir: ./data/output
sample_rate: 24000
mp3_sample_rate: 24000
lead_silence_ms: 200
default_voice: af_heart
save_wav: false
simplify_punctuation: false
n_threads: 8
timeout_seconds: 120
host: 0.0.0.0
port: 12000

Voices

The no-espeak Kokoro model exposes these voice identifiers:

af_alloy, af_aoede, af_bella, af_heart, af_jessica, af_kore, af_nicole,
af_nova, af_river, af_sarah, af_sky, am_adam, am_echo, am_eric,
am_fenrir, am_liam, am_michael, am_onyx, am_puck, am_santa, bf_alice,
bf_emma, bf_isabella, bf_lily, bm_daniel, bm_fable, bm_george

Usage

Interactive Chat (REPL)

just chat

Starts an interactive terminal REPL that synthesizes and plays each submission with the local TTS.cpp tts-cli. Generation for the next line overlaps playback of the current one, so there is no gap between utterances. The REPL drives the shared TTS runtime directly and does not require the FastAPI server to be running.

If --voice is not supplied, the REPL prompts you to pick a voice; otherwise it uses the one you pass. It reads settings (model, sample rate, save_wav, and more) from config.yaml and fails immediately if the configured tts-cli or GGUF model is missing.

Text File to MP3

just convert af_heart data/input/story.txt data/output/story.mp3

Synthesizes a text file with the same pipeline as just chat and writes the result as an MP3 instead of playing it. Each blank-line-separated paragraph is handled like one chat submission; the paragraphs are concatenated into a temporary WAV, which ffmpeg encodes as a mono MP3 at mp3_sample_rate (24 kHz, Kokoro's native rate) with LAME variable bitrate quality 7 (-q:a 7); the WAV is then deleted. Requires ffmpeg on PATH. The output path must end in .mp3; its parent directories are created. If the output file already exists, the conversion is skipped — delete the file to regenerate it.

Input controls:

Run it directly with uv run for more options:

# Pick a voice interactively, then type lines to speak
uv run -m src.main

# Skip voice selection
uv run -m src.main --voice af_heart

# One-shot: synthesize a single string and exit
uv run -m src.main --voice am_adam "Hello from Kokoro."

# List previously generated WAV files in data/output/ and exit
uv run -m src.main --list-outputs

When save_wav: true, each utterance is written to a timestamped WAV under output_dir; when false, audio is played and the temporary file is removed.

Server

just start

Starts a FastAPI server with queued playback. The server validates config.yaml at startup and processes requests sequentially through a background worker.

API

FastAPI auto-generates interactive docs at /docs (Swagger) and /redoc (ReDoc) when the server is running.

POST /say

{
  "text": "Hello, this is a Kokoro TTS request.",
  "voice": "af_heart"
}

Returns 202 Accepted with a message ID and queue position:

{
  "message_id": "msg_20260627_130430_001",
  "status": "queued",
  "queue_position": 0
}

Audio plays through the server machine's speakers.

Message Lifecycle

queued -> generating -> playing -> completed

Failures are reported as:

error

Completed and failed statuses are evicted lazily after 1 hour when later /say or /status requests trigger status cleanup.

MCP Server

The MCP server (mcp/tts-mcp.ts) is a transparent relay between MCP clients and the FastAPI server. It exposes three tools:

Setup

just mcp-install

Usage with Claude Code / Claude Desktop

Start the FastAPI server first:

just start

Then configure the MCP client to run the TypeScript relay directly. For Claude Code, from the project directory:

claude mcp add --scope local kokoro-tts-project \
  -e KOKORO_TTS_CONFIG_PATH=/path/to/low-latency-tts-service-mcp/config.yaml \
  -- /path/to/low-latency-tts-service-mcp/mcp/node_modules/.bin/tsx \
  /path/to/low-latency-tts-service-mcp/mcp/tts-mcp.ts

For JSON-based MCP configuration:

{
  "mcpServers": {
    "kokoro-tts-project": {
      "command": "/path/to/low-latency-tts-service-mcp/mcp/node_modules/.bin/tsx",
      "args": ["/path/to/low-latency-tts-service-mcp/mcp/tts-mcp.ts"],
      "env": {
        "KOKORO_TTS_CONFIG_PATH": "/path/to/low-latency-tts-service-mcp/config.yaml"
      }
    }
  }
}

The MCP relay reads host and port from config.yaml and calls /health before tool requests. Successful FastAPI JSON responses are returned as MCP text content; health check failures and non-OK HTTP responses are wrapped as structured MCP error results.

Development

Code Quality

Testing

CI

• just ci - Run all validation checks (verbose)

• just ci-quiet - Run all checks (silent, fail-fast)

The CI pipeline runs in order: init, code-format, code-style, code-typecheck, code-security, code-deptry, code-spell, code-semgrep, code-audit, test, code-architecture, code-lspchecks, and mcp-typecheck.

AI-Assisted Development

This project includes an AGENTS.md file with development rules for AI coding assistants.