Voice MCP Server
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Voice MCP Server (TTS, STT)
1. Overview
Give your AI agents a voice and a way to listen. Voice MCP Server turns recordings into timestamped transcripts and text into audio using Whisper and Qwen3-TTS on your own hardware. Choose a preset voice or clone one from a short reference recording.
Transcription and synthesis run as asynchronous jobs. Agents can submit work in the background, receive a job ID, and poll jobs for progress and results. PostgreSQL preserves job state across client disconnects and worker restarts.
Run everything in one process for local use, or keep the MCP server online on CPU or seperate on-demand GPU worker when jobs arrive.
Related MCP server: AudioPod MCP Server
2. Features
Speech-to-text produces timestamped transcripts and subtitles from individual files or batches.
Text-to-speech generates audio with a built-in speaker or a voice cloned from an authorized reference recording.
Authentication validates OIDC-issued bearer JWTs and the
mcp:accessscope on Streamable HTTP. Jobs and voices are scoped to the caller. Localstdiocalls run under the OS user.Durable jobs support background submission, status and progress polling, retries, and recovery of interrupted work.
GPU on demand starts a separate worker for queued jobs or scheduled operating periods, then drains and stops it when idle. Running the worker for 10% of low-demand time cuts its runtime resources by 90%.
Local or S3 storage holds transcripts, audio outputs, and voice references.
Extensible adapter architecture provides interfaces for integrating additional speech models, inference backends, storage providers, and GPU worker targets.
2.1. Platform profiles
Profile | Speech-to-text | Text-to-speech |
| faster-whisper on CPU with INT8 | Qwen3-TTS with PyTorch on CPU |
| faster-whisper on NVIDIA CUDA | Qwen3-TTS with PyTorch on CUDA |
| mlx-whisper on Apple Silicon | Qwen3-TTS via mlx-audio |
The server selects the backend automatically. Use health_check or python -m app diagnose to see the selected profile, devices, and installed models.
2.2. Text-to-speech (TTS)
Call text_to_speech with text and a voice_id from list_voices. Preset speakers use the Qwen3-TTS CustomVoice checkpoint. Cloned voices use the Base checkpoint and a reference prepared by register_voice, which can apply DeepFilterNet denoising. Cloning on MLX requires the matching reference transcript.
SynthesisService splits long text into chunks, synthesizes and stitches them, then uses FFmpeg to master and encode the result as WAV, MP3, FLAC, or OGG. The completed job returns the audio location. TTS pipeline details
2.3. Speech-to-text (STT)
Call speech_to_text with an allowed audio file. TranscriptionService decodes it with FFmpeg, optionally denoises it with DeepFilterNet, detects speech with Silero VAD when enabled, and runs Whisper. Results include a text preview and TXT, JSON, SRT, and VTT files. JSON includes segment timing and word timestamps when enabled.
Choose a preset (fast, balanced, accurate, or studio) or set options such as denoise per request. transcribe_batch creates a transcription job for each audio file and publishes a manifest when the batch finishes. STT pipeline details
2.4. GPU on demand
API accepts requests while the GPU worker is stopped. It validates jobs against the worker's last published capabilities and stores them in PostgreSQL. PostgreSQL LISTEN/NOTIFY signals a running worker, and recovery polls pick up missed job.
WakeController starts the worker when runnable jobs arrive or a scheduled warm window begins. The default idle timeout is five minutes. During drain, the worker stops claiming jobs and finishes the current one. If the drain deadline expires, the job goes back to the queue.
One inference worker is active per database. The controller starts and stops a Docker container or sets the ECS worker service's desired count to zero or one. A managed ECS capacity provider can also scale in the underlying GPU EC2 instances when the Auto Scaling group allows a minimum of zero. Worker lifecycle · ECS capacity providers
3. Prerequisites
Native setup uses Python 3.14, uv, FFmpeg with
ffprobe, and PostgreSQL 16 or newer.Docker deployment uses Docker Engine and Compose v2. NVIDIA deployments also need a compatible host driver and the NVIDIA Container Toolkit.
Model installation needs access to Hugging Face or a local copy of the weights and disk space for models and audio assets.
Streamable HTTP requires an OIDC provider. Run the included Keycloak realm with Compose for the local HTTP demo.
Demo scripts require Bash,
curl,jq, andffprobe.
4. Project layout
.
├── app/
│ ├── adapters/ # Models, FFmpeg, identity, storage, and compute
│ ├── config/ # Defaults, model registry, and presets
│ ├── db/ # PostgreSQL models and repositories
│ ├── domain/ # Job, audio, and voice types and rules
│ ├── mcp_server/ # MCP tools, transport, and caller identity
│ ├── migrations/ # Alembic migrations
│ ├── ports/ # Interfaces for infrastructure adapters
│ ├── services/ # Transcription, synthesis, voices, and job submission
│ ├── workers/ # Job execution, recovery, and GPU wake control
│ ├── cli.py # Commands for setup, serving, diagnostics, and worker control
│ └── main.py # Connects application components and runs the MCP server or worker
├── deploy/ # Docker Compose, Keycloak, and AWS guidance
├── docs/ # Architecture and pipeline documentation
├── scripts/ # MCP client smoke tests
├── tests/ # Unit, contract, integration, and real-model tests
├── pyproject.toml # Package metadata and tooling
└── uv.lock # Locked dependencies5. Installation and configuration
5.1. Clone and configure
git clone https://github.com/eugene-author/voice-mcp-server.git
cd voice-mcp-server
cp .env.example .envFor a native install, you need a PostgreSQL login that matches DATABASE_URL. This command starts a server that matches the template:
docker run -d --name voice-postgres -p 127.0.0.1:5434:5432 \
-e POSTGRES_USER=voice -e POSTGRES_PASSWORD=change-me -e POSTGRES_DB=voice postgres:16.6If you use your own server, put its login in DATABASE_URL. python -m app migrate --create-database creates the database if it is missing and the login is allowed to; plain migrate only applies migrations and reports a missing database.
For local Keycloak, set KEYCLOAK_ADMIN_PASSWORD in .env. For the Compose PostgreSQL service, set POSTGRES_PASSWORD.
Setting | Purpose |
| PostgreSQL connection for jobs, voices, and worker state. |
| Runtime data directory. Defaults to |
|
|
| JSON list of absolute input directories. Defaults to |
| Issuer URL exactly as it appears in JWTs. |
| Both set to the MCP endpoint URL that clients use. |
|
|
See the environment template and configuration defaults for the full settings.
5.2. Install dependencies and models
Install Python with uv python install 3.14, then select the dependency extras for your host.
Host | Command |
Linux CPU |
|
Linux NVIDIA (CUDA) |
|
Apple Silicon |
|
Run migrations, install the default Whisper model and the Qwen3-TTS cloning checkpoint, and check the setup.
uv run --no-sync python -m app migrate --create-database
uv run --no-sync python -m app models install --stt small
uv run --no-sync python -m app models install --tts Qwen3-TTS-12Hz-0.6B-Base
uv run --no-sync python -m app diagnoseInstall --tts Qwen3-TTS-12Hz-0.6B-CustomVoice for preset speakers and --denoise deepfilternet3 for optional noise reduction using the same models install command. Use --from /path/to/weights to install from local weights. Inference uses the installed weights offline.
5.3. Start the MCP server
Configure a local MCP client to launch this command from the repository root for stdio.
uv run --no-sync python -m app serve --transport stdioFor authenticated HTTP and the demo, start the reference Keycloak realm, then run the native server over Streamable HTTP.
docker compose --env-file .env -f deploy/compose.yaml --profile auth up -d --wait keycloak
uv run --no-sync python -m app serve --transport streamable_httpConnect at http://127.0.0.1:8000/mcp. The realm includes voice-mcp-interactive for interactive clients with PKCE and voice-mcp-agent for clients using the client credentials flow. Use an access token with audience http://127.0.0.1:8000/mcp and scope mcp:access. For a remote HTTP deployment, configure the public MCP URL and the matching Host and, for browser clients, Origin allowlists. Reference Keycloak realm
5.4. Separate GPU worker on demand
The API and the worker can run as separate processes that share one PostgreSQL database. The API accepts jobs and the worker runs them. You can start the worker manually, or let the API start and stop it on demand.
Start the worker manually
Stop any server running with the default all role first. It holds port 8000 and the single worker lock.
On the worker host, after installing weights, register the worker once. This lets the API accept jobs while the worker is stopped. Repeat it whenever the installed models change.
uv run --no-sync python -m app worker --register-onlyThen start each process in its own terminal:
uv run --no-sync python -m app serve --role api
uv run --no-sync python -m app workerJobs submitted while the worker is stopped stay queued, and it runs them when it starts.
Start and stop it on demand
The startup script runs the MCP API on the host and manages a worker container through the local Docker socket.
From the repository root, start PostgreSQL and Keycloak, wait for the voice realm, and launch the API.
docker start voice-postgres voice-mcp-keycloak-1
until curl -fs -o /dev/null --max-time 5 \
http://127.0.0.1:8080/realms/voice/.well-known/openid-configuration; do
sleep 2
done
bash scripts/start_on_demand.shThe script applies database migrations, builds the Python 3.14 worker image with FFmpeg if needed, and creates a stopped container. The worker uses host networking and the invoking user's UID and GID. The script mounts the checkout and host Python installation at their original paths, so the worker can use the host's .venv and models. The script needs a .venv built with a uv-managed or pyenv Python, because it mounts that installation into the worker container; it stops with instructions if .venv uses a system Python.
The script runs worker --register-only on the host to publish the installed model capabilities, then starts the Streamable HTTP API in the foreground at http://127.0.0.1:8000/mcp by default. The controller starts the container for queued jobs or scheduled operating periods. Outside scheduled periods, it requests a drain once the idle timeout and minimum uptime have both elapsed.
For a shorter test, run FAST=1 bash scripts/start_on_demand.sh.
Docker on one NVIDIA host
Set absolute INPUTS_DIR and ASSETS_DIR paths in .env and set DOCKER_GID to the Docker socket's group ID. Give the container user (UID 10001) read access to inputs and write access to the model volume for installation, workspace volumes, and the assets directory. Base Compose file · CUDA overlay
Use the Compose admin service to install weights into the shared model volume. Register the worker before starting the API with on-demand control enabled.
compose_gpu() {
docker compose --env-file .env \
-f deploy/compose.yaml -f deploy/compose.gpu.yaml \
--profile auth --profile split "$@"
}
compose_gpu up -d --wait postgres keycloak
compose_gpu run --rm migrate
compose_gpu --profile tools run --rm -e HF_HUB_OFFLINE=0 -e TRANSFORMERS_OFFLINE=0 \
admin models install --stt small --backend ctranslate2
compose_gpu --profile tools run --rm -e HF_HUB_OFFLINE=0 -e TRANSFORMERS_OFFLINE=0 \
admin models install --tts Qwen3-TTS-12Hz-0.6B-Base --backend torch
compose_gpu create gpu-worker
compose_gpu run --rm --no-deps gpu-worker worker --register-only
GPU_ON_DEMAND=true compose_gpu up -d api docker-proxyThe socket proxy gives the controller access to inspect, start, and stop the worker container. Compose setup and operator commands
AWS ECS across hosts
Run the API on CPU and the worker as a separate ECS service on GPU EC2. Set STORAGE_BACKEND=s3 with the same bucket and region on both services. Install the aws and s3 extras on the API and grant its task role ecs:DescribeServices and ecs:UpdateService on the worker service. The API ingests input audio into S3 so the two hosts can share assets.
SERVICE_ROLE=api
GPU_ON_DEMAND=true
VOICE_MCP_STORAGE__INPUT_TRANSFER=ingest
VOICE_MCP_GPU_WORKER__ON_DEMAND__TARGET=aws_ecs
VOICE_MCP_GPU_WORKER__ON_DEMAND__AWS_ECS__REGION=eu-west-1
VOICE_MCP_GPU_WORKER__ON_DEMAND__AWS_ECS__CLUSTER=voice
VOICE_MCP_GPU_WORKER__ON_DEMAND__AWS_ECS__SERVICE=voice-gpu-workerUse health_check or python -m app gpu status to see the worker state and controller decisions. AWS configuration and IAM example
6. MCP tool reference
Use tools/list to discover the full argument schemas. The table shows the inputs you'll use most often.
Tool | Main arguments | Result |
| None | Server, database, storage, runtime, and worker status. |
|
| Models by backend, with install and runtime status. |
|
| Available preset speakers and the caller's cloned voices. |
|
| Registration job and |
|
| Transcription job with a text preview and transcript files on completion. |
|
| Synthesis job with the rendered audio location on completion. |
|
| Parent job with child progress and a final |
|
| Job status, progress, and output locations. Presigned URLs require S3 and presigning enabled. |
| Optional | Newest jobs first, scoped to the caller, with cursor pagination. |
Use wait: false on job submissions to return immediately, then poll get_job with the returned ID. Use a stable request_id when retrying the same submission to avoid creating a duplicate job. Input arguments are absolute paths visible to the server.
7. Example requests and usage
7.1. Run the demo
With the HTTP server and Keycloak running, open a second terminal at the repository root. With the default data root, copy the included recording and transcript to a demo subdirectory.
demo_inputs="$PWD/var/inputs/demo"
mkdir -p "$demo_inputs"
cp tests/fixtures/demo/jessica.wav tests/fixtures/demo/jessica.txt "$demo_inputs/"
chmod +x scripts/0[1-4]_*.sh
bash scripts/run_all.shScript | What it shows |
| Check token enforcement, open an MCP session, list tools, and check health. |
| Register the reference voice and save its ID when ready. |
| Transcribe the recording and verify the output files. |
| Synthesize a new sentence with the saved voice and verify the audio. |
The scripts read the local Keycloak admin password from .env. Set MCP_CLIENT_SECRET to use an existing client secret. Demo script options
7.2. Call the tools directly
In an initialized MCP session, pass these objects as params to tools/call. Replace the example paths with absolute paths inside an allowed input directory.
{
"name": "speech_to_text",
"arguments": {
"file_path": "/absolute/path/to/voice-mcp-server/var/inputs/demo/jessica.wav",
"preset": "balanced",
"wait": false
}
}To synthesize speech with a cloned voice, first register a reference recording you have permission to use. This example uses the demo recording and its exact transcript.
{
"name": "register_voice",
"arguments": {
"name": "Jessica",
"reference_audio_path": "/absolute/path/to/voice-mcp-server/var/inputs/demo/jessica.wav",
"reference_text": "Hello, it's nice to meet you. I would like to tell you a great story today.",
"language": "en",
"consent_confirmed": true,
"wait": false
}
}When the registration job reaches completed, use the voice_id returned by register_voice in the synthesis request.
{
"name": "text_to_speech",
"arguments": {
"text": "The voice worker is ready.",
"voice_id": "<voice-id>",
"options": {"output_format": "wav"},
"wait": false
}
}Poll get_job with the returned job_id for any submission. Completed transcription and synthesis jobs include output locations in files.
{"name": "get_job", "arguments": {"job_id": "<returned-job-id>"}}8. Tests
uv run --no-sync pytest
TEST_DATABASE_URL='postgresql://voice:password@127.0.0.1:5433/voice_test' \
uv run --no-sync pytest -m "integration or s3"Use a disposable PostgreSQL database for integration tests because they truncate application tables. S3 tests use a simulated bucket through Moto. The real-model suites need the corresponding dependency extras, FFmpeg, and installed weights.
Suite | Command |
Linux CPU models |
|
Apple Silicon models |
|
NVIDIA CUDA models |
|
9. License
The server code is released under the MIT License. Dependencies and model weights are distributed under their respective upstream licenses.
10. Disclaimer
The software is provided as is. Use voice cloning only with the speaker's consent and process recordings you have permission to use. Review generated audio and transcripts before publishing or relying on them.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Audio for your agent: transcribe, speak, translate, summarise, plus sound effects and music.
Audio and video tools for your AI assistant: remove noise, enhance voices, transcribe, and more.
Transcribe audio & video to text for AI agents: 100+ languages, speaker labels, webhooks.
Text to speech and transcription for any AI model: MP3 voiceovers, audio and YouTube to text.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to perform comprehensive audio processing—stem separation, analysis, transcription, restoration, speech processing, and generation—through a unified self-hosted API with asynchronous jobs and webhooks.6Do What The F*ck You Want To Public

AudioPod MCP Serverofficial
AlicenseNot gradedqualityCmaintenanceProvides ten hosted audio AI tools — TTS, voice cloning, music generation, stem separation, speaker separation, transcription, denoising, media conversion, and job polling — over a single streamable-HTTP endpoint for any MCP-capable agent.MIT- AlicenseNot gradedqualityBmaintenanceEnables users to generate speech, music, cloned voices, lyrics, and separated vocal/instrumental stems through 50+ audio model APIs from providers such as MiniMax, Suno, Mureka, Doubao, and Qwen TTS.MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to transcribe and summarize videos and voice messages, translate speech while keeping the speaker's voice, read text aloud in a cloned voice, and produce subtitles, lip-synced dubbing, voice changes, and translated images from links, uploads, or local files. It also handles media downloading and job/account status, with paid operations priced and confirmed before they run.MIT