YouTube MCP Server

A Python-based Model Context Protocol (MCP) server that extracts educational content from YouTube videos — transcripts and visually meaningful frames — and makes it available to AI assistants like Claude, ChatGPT, and any MCP-compatible client.

What It Does

Give an AI assistant a YouTube URL, and it can:

1. Read the transcript — Full captions with timestamps, ready for summarization, Q&A, or content analysis.

2. See the video — A 5-phase computer-vision pipeline extracts the most informative frames, scores them for quality, removes duplicates, and returns them as base64-encoded JPEGs the AI can "look at."

No manual downloading. No copy-pasting. Just paste a URL and ask questions.

Tools

get_transcript

Fetches the video's captions using youtube-transcript-api.

Parameters:

• url (required) — YouTube video URL

• language (optional, default: "en") — Preferred caption language

• prefer_manual (optional, default: true) — Prefer manually-written captions over auto-generated

Returns: A clean, timestamped transcript block:

[00:00:00] So I want to start by offering you a free ...
[00:00:15] The key insight here is that ...

get_video_frames

Runs a full video-analysis pipeline to surface the most meaningful visual moments.

Parameters:

• url (required) — YouTube video URL

• max_frames (optional, default: 20) — Maximum frames to return (hard cap: 40)

• scene_threshold (optional, default: 0.25) — FFmpeg scene-detection sensitivity

• output_width (optional, default: 640) — Width of returned JPEGs (proportional height)

• min_importance_score (optional, default: 0.35) — Minimum composite quality score

Returns:

• Metadata: video_id, duration_seconds, pipeline_stats, fallback_used

• Frame blocks: each includes timestamp_ms, composite_score, and a base64-encoded JPEG image

• An index listing every returned frame with its timestamp and score

The Frame Pipeline (5 Phases)

Quick Start

Prerequisites

• Python 3.11

• ffmpeg — video processing

• Tesseract OCR — text detection in frames

macOS:

brew install ffmpeg tesseract

Ubuntu / Linux:

sudo apt-get install ffmpeg tesseract-ocr tesseract-ocr-eng

Windows: Download ffmpeg and Tesseract, then add both to your PATH.

Verify:

ffmpeg -version
tesseract --version

Installation

# Clone the repository
git clone https://github.com/Ayush-Mamgain/youtube-mcp-server.git
cd youtube-mcp-server

# Create a virtual environment
python3 -m venv venv
source venv/bin/activate        # macOS / Linux
# venv\Scripts\activate      # Windows

# Install dependencies
pip install -r requirements.txt

Configuration

Create a .env file in the project root:

LOG_LEVEL=DEBUG
MCP_HTTP_PORT=8000

Optional tuning (loaded from config.py defaults if omitted):

Start the Server

python server.py

The server starts at http://localhost:8000.

Connecting to Claude.ai (Local)

1. Start the server: python server.py

2. Go to Claude.ai → Settings → Integrations → Add MCP server

3. Enter: http://localhost:8000/mcp

For cloud-hosted Claude to reach your server, you'll need to expose it publicly (see Deployment below).

API Endpoints

Project Structure

youtube-mcp-server/
├── server.py              # MCP entry point — FastMCP + Starlette HTTP server
├── config.py              # Loads and validates environment variables
├── logger.py              # stderr-only logging
├── url_parser.py          # Validates YouTube URLs and extracts video IDs
├── transcript.py          # Fetches captions via youtube-transcript-api
├── downloader.py          # Phase 1 — video download with yt-dlp
├── frame_extractor.py     # Phase 2 — scene-change frame extraction via ffmpeg
├── scorer.py              # Phase 3 — multi-signal frame scoring (OpenCV + Tesseract)
├── deduplicator.py        # Phase 4 — perceptual-hash deduplication
├── frame_loader.py        # Phase 5 — resize, JPEG encode, base64
├── video_frames.py        # Orchestrates Phases 1–5 with semaphore and cleanup
├── requirements.txt       # Pinned Python dependencies
└── .gitignore             # Excludes .env, venv, caches, test artifacts

Development Workflow

This project was built in 9 self-contained stages, each with its own test file and verification step:

Golden rule: Each stage is verified before proceeding. Run python test_stage{N}.py to validate.

Deployment

Docker support is planned but not yet configured. This section will be updated once containerization is complete.

For now, the server runs directly on any machine with Python 3.11, ffmpeg, and Tesseract installed. Recommended hosting options once Docker is ready:

• Railway

• Fly.io

• Render

Once deployed, update your Claude.ai integration URL to:

https://YOUR-DEPLOYMENT-URL/mcp

Design Notes

• All logging goes to stderr only. stdout is reserved exclusively for MCP communication.

• Thread-safe: The get_video_frames pipeline uses a threading.Semaphore(1) to prevent concurrent downloads from overwhelming the system.

• Automatic cleanup: Temporary files created during video processing are deleted after each run.

• Strict validation: Video IDs are validated against ^[A-Za-z0-9_-]{11}$. Playlist-only URLs, channel URLs, and malformed inputs are rejected with clear error messages.

• Graceful degradation: If all frames score below the minimum importance threshold, the pipeline falls back to the top 5 frames and flags fallback_used: true.

License

MIT © Ayush Mamgain

Acknowledgments

Built with FastMCP, Starlette, yt-dlp, youtube-transcript-api, and OpenCV.