mcp-youtube-transcribe

An MCP server that hands an AI client the text of a YouTube video: its metadata, the captions YouTube already has, or a Whisper transcription run on the local GPU.

Six tools, cheapest first. Bounded returns that survive a context window. Background jobs that survive a request timeout. One GPU, serialised.

The problem

An assistant asked about a YouTube video cannot watch it. The obvious fix is a tool that downloads the audio and runs Whisper over it, and that tool is about forty lines. The forty-line version then fails in three separate ways the first time it meets a real video, and none of the three is about speech recognition.

• The transcript is bigger than the context window. A tool's return value goes straight into the model's context, and a transcript is long: the eighteen-minute talk shown below is 18,430 characters of captions, so a three-hour one runs to a couple of hundred thousand. Returning "the transcript" is the one thing this tool must not do. So every text-returning tool hands back a bounded slice, the true total length, and the offset needed to ask for the next one, while the full text stays on disk where paging is free.

• The work outlives the request. A long video is minutes of GPU time, and an MCP client's request timeout is much shorter than that. A tool that blocks until it finishes is a tool that times out and throws away work it had already done. So transcribe_video splits on duration: short videos run inline and return their text, long ones return a job_id immediately and finish in the background, and the transcript lands on disk either way.

• The expensive path is usually the wrong one. Most YouTube videos already carry captions, and fetching them is a metadata request that costs no GPU at all. A single transcribe(url) tool would download the audio and put the graphics card to work reproducing a file that was already sitting there. So the surface is shaped to make the model check first.

That third one is the reason there are six tools rather than one. get_video_info downloads nothing and reports, among other things, exactly which caption languages exist:

{
  "video_id": "aircAruvnKk",
  "title": "But what is a neural network? | Deep learning chapter 1",
  "uploader": "3Blue1Brown",
  "duration_sec": 1120,
  "duration_string": "18:40",
  "upload_date": "20171005",
  "view_count": 24249422,
  "is_live": false,
  "chapters": [
    {"title": "Introduction example", "start_time": 0, "end_time": 67},
    {"title": "Series preview", "start_time": 67, "end_time": 162},
    ...
  ],
  "subtitles_manual": ["ar", "bn", "cs", "de", "el", "en", "es", ...],
  "subtitles_auto": ["aa", "ab", "af", "ak", "am", "ar", "as", ...]
}

Thirty manual caption languages and a hundred and fifty-seven auto-generated ones, and the model can see that before it commits to anything. get_subtitles then returns the same eighteen minutes of speech with no download and no GPU, in the envelope every text-returning tool here uses:

{
  "available": true,
  "language": "en",
  "kind": "manual",
  "text": "This is a 3.\nIt's sloppily written and rendered at an extremely low resolution of 28x28 pixels,\nbut your brain has no trouble recognizing it as a 3.\n...",
  "total_chars": 18430,
  "truncated": true,
  "note": "Showing the first 6000 of 18430 characters. Use read_transcript(video_id) for the complete transcript."
}

Whisper is the fallback, not the default. It is what you reach for when the captions do not exist, when they are auto-generated garbage, or when you want timings you can trust.

Related MCP server: YouTube Content Extractor MCP

Features

• Six tools, ordered cheapest first: metadata that downloads nothing, then captions that cost one more metadata request, then transcription that costs the GPU, with each tool's description telling the model when the next one down is the better move

• Bounded, pageable returns: every text response carries total_chars, truncated and the on-disk path, and read_transcript(video_id, offset=...) walks the rest by character offset

• A duration threshold, not a timeout: videos under SYNC_LIMIT_SEC transcribe inline, longer ones return a job_id and report progress through get_job_status

• Real progress, not a spinner: percentages come from the position of the segment currently being decoded, so a two-hour job is legible while it runs

• A transcript cache keyed on video and model: a repeat call returns from disk, a call for a format that was never written falls through and re-transcribes, and force=True overrides both

• One GPU, one transcription: downloads and transcodes overlap freely, but the Whisper step is behind a semaphore of one, because two large-v3 loads exhaust a 10 GB card

• Four output formats: txt, srt, vtt and json, written next to a meta.json and re-readable without touching the GPU again

• A long-lived container, not a spawned subprocess: streamable HTTP rather than stdio, so 3 GB of model weights load once and stay resident across clients and sessions

• A Host-header allow list: DNS-rebinding protection built at startup from the loopback defaults plus whatever you configure, and logged so you can see what will be accepted

• Two documented remote-access routes: a Tailscale overlay that publishes nothing, and a Cloudflare tunnel with Access in front of it

• Media is never kept: audio goes to a per-call temp directory removed in a finally, so a failed job cannot quietly eat the disk

• CPU fallback on the same image: comment out one deploy: block and the identical container runs int8 on the CPU

Quick start

Docker with Compose. An NVIDIA GPU is optional but strongly wanted: without one the server runs int8 on the CPU, correctly and slowly. GPU passthrough needs a CDI spec generated first, which is docs/operations.md.

git clone https://github.com/tintando/mcp-youtube-transcribe
cd mcp-youtube-transcribe

cp .env.example .env          # optional: every setting has a working default
docker compose up -d --build
docker compose logs -f

The startup line is the thing to read, because it is the only reliable way to tell a GPU run from a silent CPU fallback:

2026-09-01 01:00:35,151 INFO    yt-transcribe: allowed hosts: 127.0.0.1:8765, localhost:8765
2026-09-01 01:00:35,449 INFO    yt-transcribe: starting yt-transcribe on 0.0.0.0:8765 | model=large-v3 device=cuda compute=float16
INFO:     Uvicorn running on http://0.0.0.0:8765 (Press CTRL+C to quit)

Then point a client at it:

claude mcp add --transport http yt-transcribe http://127.0.0.1:8765/mcp --scope user

User scope makes it available from every directory. The first transcription downloads large-v3, about 3 GB, into data/models/, which is a bind mount, so it happens once and survives rebuilds.

Without a GPU

Comment out the whole deploy: block in compose.yaml. Nothing else changes: the image is the same, detect_device() finds no CUDA device and falls back to int8 on the CPU, and the startup line says device=cpu.

Checking it end to end

uv run --with 'mcp[cli]' scripts/smoke.py

Connects over HTTP, lists the tools, pulls metadata and chapters, transcribes a short clip cold, checks the files landed on the host owned by you rather than by root, repeats the call to prove the cache hit, and confirms a non-YouTube URL is refused. It transcribes for real, so it needs the network and some patience; --skip-transcribe stops after the metadata checks.

Configuration

Everything is an environment variable, and .env.example carries all of them with the reasoning next to each. Compose reads .env automatically.

Remote access, both the Tailscale overlay and the Cloudflare tunnel, is docs/remote-access.md. The short version is that this server has no authentication of its own, so it binds to loopback and everything else is a deliberate act.

How it works

The tool surface, which is the part a client actually sees:

A long-lived container, not a subprocess. The default MCP transport is stdio: the client spawns the server as a child process, and it dies with the session. That is the wrong shape here. large-v3 is about 3 GB of weights that take real time to get onto the card, and a stdio server would pay that on every session. So this one speaks streamable HTTP, runs under Compose with restart: unless-stopped, and keeps a single WhisperModel in a module-level cache keyed on model name, device and compute type. It loads on the first transcription and stays resident for every client afterwards. The price of that choice is a network surface, which is where the loopback binding and the Host-header allow list come from.

The GPU is a semaphore; the blocking work is a thread. faster-whisper's transcribe() blocks, so calling it on the event loop would freeze every other tool for the length of a transcription. Every blocking step, yt-dlp, ffmpeg, Whisper and the file writes, goes through anyio.to_thread.run_sync, and the loop keeps answering get_job_status and list_transcripts throughout. Only the Whisper call sits inside asyncio.Semaphore(1): downloads and transcodes are free to overlap, but two large-v3 loads at once do not fit in 10 GB.

Progress comes out of the segment generator. faster-whisper hands back a generator and does the decoding lazily as you consume it, so there is no progress callback to register. The loop that collects segments divides seg.end by the known duration as it goes, which is why a long job reports a moving percentage rather than nothing at all. It is mapped into a 0.12 to 0.97 band, leaving the ends for the download and the write, so a job at 5% is genuinely still fetching audio.

One place decides how much text leaves. _clip() is the only path text takes out of the server. Under RETURN_CHARS it returns everything and says truncated: false. Over it, the leading slice, the true total_chars, the on-disk path and a note naming the tool that pages. read_transcript then takes a character offset and returns next_offset, so a client walks a long transcript without guessing where to cut.

The sync/async split is a duration, decided before any download. transcribe_video extracts metadata first, which is cheap, and compares the real duration to SYNC_LIMIT_SEC. Under it, the pipeline runs inline and the text comes back in the tool result. Over it, the same coroutine is wrapped in a Job, started as a task on the running loop, and a job_id returns immediately with a message telling the model to poll. Jobs are a dict capped at 100, evicting finished ones first, and they do not survive a restart. The transcript is written to disk either way, so a lost job costs the id, not the work.

The cache is keyed on video and model, and admits when it misses. Each transcript directory carries a meta.json recording the model that produced it, so a repeat call with the same model reads the text off disk and never wakes the GPU. A call asking for vtt when only txt and srt were written falls through and re-transcribes rather than reporting a hit it cannot honour, and force=True skips the check entirely. The lookup happens after the metadata extraction rather than before it, which costs every cache hit one yt-dlp round trip; see the gaps below.

Refusals happen before the download, not after. validate_url accepts six YouTube hostnames and nothing else, so the tool cannot be talked into fetching an arbitrary URL on the model's behalf. check_duration then rejects anything over MAX_DURATION_SEC before a byte moves, which is what keeps a mis-pasted livestream VOD from filling the disk. yt-dlp's error zoo is translated on the way out: the whole bot-check family becomes one message about exporting cookies, aimed at the person who can act on it, rather than a stack trace the model will try to reason about.

Captions are cleaned, not concatenated. YouTube's auto-captions are a rolling two-line window, so most lines arrive twice, once as the new line and once as the top line of the next cue. Naive joining doubles the text. vtt_to_text drops cue numbers, timecodes, inline tags and the WEBVTT preamble, then drops consecutive duplicates, then drops any line wholly contained in the one before it, which is the exact shape the rolling window leaves behind.

Audio is a temp directory, not a download folder. yt-dlp fetches bestaudio into a per-call temp directory and ffmpeg transcodes it to 16 kHz mono WAV, which is Whisper's native rate, so the resample happens once in ffmpeg instead of inside the model on every run. The directory is removed in a finally, so a crash mid-transcription leaves nothing behind. Only transcripts persist.

Writes land as you, not as root. The Dockerfile creates its runtime user at UID and GID 1000 to match the host user, and both bind mounts carry :z so SELinux relabels them. Skip the first and every transcript is root-owned; skip the second and an enforcing Fedora host gives the container EACCES on every write. The smoke test asserts the resulting ownership rather than just the existence of the files, because that is the check that actually catches either mistake.

Scope and known gaps

• No test suite. CI installs the package, runs ruff check and ruff format --check, then imports the server and asserts that all six tools still register. That is a lint and import gate, and it is worth saying plainly rather than dressing up. scripts/smoke.py is the real end-to-end check, and it needs a running container, a GPU and the network, so it cannot run there.

• No authentication of any kind. The mitigation is that it binds to loopback and matches on the Host header. Everything in docs/remote-access.md is about borrowing an identity layer from somewhere else, because there is none here to configure.

• get_subtitles truncates with nowhere to page to. It returns the first RETURN_CHARS characters and a note pointing at read_transcript, but captions are never written to disk, so that advice only holds for a video you also transcribed. Long captions are currently readable in full only by taking the expensive path the tool exists to avoid. Persisting fetched captions the way transcripts are persisted would fix it.

• A cache hit still pays for a metadata round trip. transcribe_video extracts video info before it consults the cache, so a transcript already on disk still waits on yt-dlp reaching YouTube. media.video_id_from_url() parses the id straight out of the URL with no network at all and would close that gap, but nothing currently calls it: it is the one piece of dead code in app/.

• Jobs die with the container. In memory, capped at 100, evicted oldest-finished-first. A restart during a long transcription loses the job and its partial work, since the transcript is only written once the run completes.

• Switching models reloads several gigabytes. Exactly one model is held, keyed on name, device and compute type, so alternating model= between calls evicts and reloads each time. Fine for a server that mostly runs one model, wasteful if you meant to compare two.

• YouTube only, and only as far as yt-dlp can still reach. Six hostnames are accepted and nothing else. Bot checks are a live failure mode rather than a theoretical one, and the mitigation is a cookie file you export yourself and treat as a credential.

• Live streams are refused, not transcribed from the live edge.

• No diarization, no translation. Segments carry timings and text, never speakers. Whisper's translate task is not exposed, and language only skips auto-detection.

• list_transcripts walks the directory and reads one meta.json per entry, sorted in memory. There is no index. That is the right amount of machinery for a personal transcript store and the wrong amount for a large one.

• The image is about 9 GB, nearly all of it the CUDA 12.8 base. It cannot drop to the plain -runtime tag: CTranslate2 4.5 and later dlopen() cuDNN 9 at runtime, so the -cudnn-runtime tag is load-bearing.

• The MCP SDK is capped below 2.x. The code targets the 1.x FastMCP API, and 2.x renamed it to MCPServer and moved the module, so an uncapped rebuild fails at import. Moving up is a port rather than a version bump.

Architecture

app/
├── server.py             # FastMCP app: the six tools, the sync/async split, the GPU semaphore
├── media.py              # yt-dlp: URL validation, metadata, caption cleaning, audio to 16 kHz WAV
├── transcribe.py         # model cache, txt/srt/vtt/json rendering, the on-disk transcript store
└── jobs.py               # in-memory registry for backgrounded transcriptions

scripts/
└── smoke.py              # end-to-end check against a running container

cloudflared/
└── config.yml.example    # tunnel ingress template; the filled-in config is gitignored

docs/
├── operations.md         # GPU passthrough via CDI, reading the startup line, the real failures
└── remote-access.md      # the Tailscale overlay and the Cloudflare tunnel, in full

Dockerfile                # CUDA 12.8 with cuDNN 9, uv, a non-root user at UID 1000
compose.yaml              # the server, loopback-bound, plus the optional `remote` tunnel profile
compose.tailscale.yaml    # overlay: a second bind on the host's tailnet address
.env.example              # every setting, with the reasoning beside it

Runtime state lives under data/, which is gitignored and never ships. data/models/ is the Whisper weight cache, about 3 GB for large-v3. data/transcripts/<video_id>/ holds transcript.txt, transcript.srt and friends next to a meta.json recording the model, language, duration, segment count and date. data/cookies.txt, if you create it, is a live Google session and is treated accordingly.

Built with

Python 3.11 or newer, the MCP Python SDK's FastMCP over streamable HTTP, faster-whisper on CTranslate2 rather than PyTorch, yt-dlp for everything YouTube-facing, ffmpeg for the one resample, and anyio for the thread offload. Ruff is the linter and formatter. No web framework of its own: FastMCP brings Starlette and uvicorn with it. No database, because a transcript is a directory with a meta.json in it and listing them is a directory walk. No task queue, because a job only has to outlive one HTTP request, and a dict and an asyncio.Task do that.

License

MIT. See LICENSE.