YouTube Dubbing MCP: AI dubbing with one cloned voice per speaker

YouTube Dubbing MCP is an open-source MCP server that lets an AI agent (Claude, Cursor, Codex, VS Code) dub a YouTube video, a local video or an SRT file into another language, giving every speaker their own cloned voice, with synced SRT captions and upload-ready YouTube multi-language audio tracks. You ask "dub this video into English and keep each speaker's voice"; the agent translates, and the server does everything that touches audio: transcript, speaker detection, voice samples, voicing, timing, mixing and packaging.

▶ Listen to the real output: English dub (MP3) · dubbed video (MP4) · original Spanish video · synced SRT · or play them in the browser on the Hugging Face demo page.

Contents

• Why this AI dubbing MCP server

• How YouTube dubbing works here

• Quick start

• Example: dub a YouTube video to English with two voices

• Tools

• Upload multi-language audio tracks to YouTube

• Comparison with other AI dubbing tools

• FAQ

• Troubleshooting

Related MCP server: OrcaDub MCP Server

Why this AI dubbing MCP server

• Multi-speaker dubbing, each voice cloned. Speakers come from caption markers (Name:, [Name], >>) or, when there are none, from voice pitch. dubbing_extract_voice_samples cuts a clean sample of every speaker from the video itself, so Anna still sounds like Anna in Spanish, German or English.

• Dub YouTube videos from a URL. Reads YouTube's manual or automatic captions when they exist (fast, free of guesswork), otherwise transcribes the audio. Local MP4/MOV/MKV/MP3/WAV files and existing SRT/VTT subtitles work too.

• Timing that fits. Each line gets a character budget for its time slot in the target language, the agent writes to it, and the renderer starts every line with the original and only speeds up (pitch kept, capped at 1.25×) when it must. The dub track is exactly as long as the video, which YouTube requires.

• Upload-ready for YouTube. One folder per language with audio.m4a (AAC 48 kHz), captions.srt, a localized title and description, and the YouTube Studio steps.

• Resumable and safe. Everything is saved in a project folder. Re-running a tool reuses finished lines, resumes pending jobs and never submits the same line twice. The API key never appears in output or logs.

• No GPU, no local models. uvx installs it in one command; ffmpeg comes bundled. Voices are rendered by the Miragea voice engine.

How YouTube dubbing works here

1. Ingest: YouTube URL → audio + captions (yt-dlp), or a local video/audio/SRT file. Caption fragments are merged into sentences; speech without captions is transcribed and re-split at pauses.

2. Speakers: names from caption markers, or grouped by voice pitch. Fix anything with one call.

3. Voices: a sample of each speaker is cut from the source and cloned, or you pick a saved voice / any public reference clip.

4. Translate: your agent's own model translates, line by line, inside the time budget. Too-long lines are reported back for shortening.

5. Render: every line is voiced in its speaker's voice, placed at its original start time, and mixed into a track of the exact video length, with an SRT and an optional preview MP4.

6. Package: per-language folders for YouTube Studio → Languages.

Quick start

1. Install uv (it provides uvx).

2. Get a voice engine API key at miragea.ai/workspace/api.

3. Add the server to your MCP client:

Claude Code

claude mcp add youtube-dubbing -s user -e MIRAGEA_API_KEY=<your-key> -- uvx youtube-dubbing-mcp

Claude Desktop / Cursor / Windsurf (claude_desktop_config.json, ~/.cursor/mcp.json)

{
  "mcpServers": {
    "youtube-dubbing": {
      "command": "uvx",
      "args": ["youtube-dubbing-mcp"],
      "env": { "MIRAGEA_API_KEY": "<your-key>" }
    }
  }
}

Codex (~/.codex/config.toml)

[mcp_servers.youtube-dubbing]
command = "uvx"
args = ["youtube-dubbing-mcp"]
tool_timeout_sec = 1800
env = { MIRAGEA_API_KEY = "<your-key>" }

VS Code (.vscode/mcp.json): run uvx youtube-dubbing-mcp --print-config vscode and paste the output.

Check everything with uvx youtube-dubbing-mcp --check (creates no job). Latest main instead of the PyPI release: uvx --from git+https://github.com/RTY2345RR/youtube-dubbing-mcp youtube-dubbing-mcp.

Example: dub a YouTube video to English with two voices

Prompt used for the demo in examples/spanish-to-english-two-speakers:

Dub interview-es.mp4 into English. Marco and Lucia should keep their own voices. Then make the YouTube upload package.

What the agent called, with the real results:

The English dub (MP3, SRT):

1
00:00:00,000 --> 00:00:00,991
[Marco] Hi everyone.

2
00:00:01,187 --> 00:00:04,815
[Marco] Today we're dubbing this video into another language in one step.

3
00:00:05,389 --> 00:00:09,059
[Lucia] And everyone keeps their own voice, with synced subtitles.

A speech-recognition pass over the finished English track reads back "Hi everyone! Today we're dubbing this video into another language in one step, and everyone keeps their own voice with synced subtitles." The male and female voices keep their pitch (about 110 Hz and 170 Hz), so each speaker stays recognisable.

Tools

Upload multi-language audio tracks to YouTube

YouTube's public Data API cannot upload additional audio tracks, so no tool can do the final click for you. The package makes it a one-minute job per language in YouTube Studio:

1. Studio → Content → your video → Languages.

2. Add language → Dub → Add → upload audio.m4a → Publish.

3. Subtitles → Add → Upload file → With timing → captions.srt.

4. Title & description → Add → paste title.txt / description.txt.

Viewers then choose the track under Settings → Audio track. Multi-language audio is available to channels YouTube has enabled for the feature.

Comparison with other AI dubbing tools

Feature status of other projects as read from their repositories and docs in November 2026; corrections welcome.

FAQ

How do I dub a YouTube video into English with AI? Add this server to Claude, Cursor or Codex and ask: "Dub https://youtu.be/… into English, keep each speaker's voice, then make the YouTube upload package." The agent runs the six steps above and you get an English audio track, SRT and upload folder.

Can AI dubbing keep the original speaker's voice? Yes. dubbing_extract_voice_samples cuts a few seconds of each speaker from the original audio and the voice engine clones it, so every speaker is dubbed in their own voice. Ask the people in the video before cloning their voices.

Can I upload multiple audio tracks to YouTube with the API? No. The YouTube Data API has no endpoint for extra audio tracks, so tracks are added in YouTube Studio → Languages. This server prepares the exact files and steps.

Is there an open-source AI video dubbing tool for multiple speakers? This project is MIT-licensed and handles any number of speakers, each with a separate voice. The voices are rendered by a hosted voice engine, so no GPU is needed.

Can I dub from an SRT file instead of a video? Yes: dubbing_start_project(source="talk.srt"). Without source audio, choose voices with dubbing_set_voice.

Is this an ElevenLabs dubbing alternative? It is an open-source dubbing workflow for AI agents with per-speaker voice cloning, editable translations and YouTube packaging. The voices come from the Miragea voice engine.

Which languages are supported? Any language your agent can translate into and the voice engine can speak. The demo is Spanish → English; per-language speaking rates are built in for 35+ languages so lines fit their slots.

Does it work offline or need a GPU? It needs internet for the voice engine and YouTube, and no GPU.

Troubleshooting

Related projects

• TTS MCP Server: text-to-speech, voiceover and SRT/VTT captions for AI agents.

• Voice Agent Skills: Agent Skills for voiceover and dubbing workflows in Claude Code and Codex.

• Live demo page with audio players.

Credits

Voices and transcription by the Miragea voice engine (https://miragea.ai/v1). Agents that support skills can also install its audio skill with: Install https://miragea.ai/skills/miragea-audio/install.md. YouTube access by yt-dlp; audio by FFmpeg.

MIT License.