# YouTube MCP Server

> A Python-based [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server that extracts educational content from YouTube videos — transcripts and visually meaningful frames — and makes it available to AI assistants like Claude, ChatGPT, and any MCP-compatible client.

[![Python 3.11](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![FastMCP](https://img.shields.io/badge/FastMCP-2.2.0+-orange.svg)](https://github.com/jlowin/fastmcp)

---

## What It Does

Give an AI assistant a YouTube URL, and it can:

1. **Read the transcript** — Full captions with timestamps, ready for summarization, Q&A, or content analysis.
2. **See the video** — A 5-phase computer-vision pipeline extracts the most informative frames, scores them for quality, removes duplicates, and returns them as base64-encoded JPEGs the AI can "look at."

No manual downloading. No copy-pasting. Just paste a URL and ask questions.

---

## Tools

### `get_transcript`

Fetches the video's captions using `youtube-transcript-api`.

**Parameters:**
- `url` *(required)* — YouTube video URL
- `language` *(optional, default: `"en"`)* — Preferred caption language
- `prefer_manual` *(optional, default: `true`)* — Prefer manually-written captions over auto-generated

**Returns:**
A clean, timestamped transcript block:
```
[00:00:00] So I want to start by offering you a free ...
[00:00:15] The key insight here is that ...
```

---

### `get_video_frames`

Runs a full video-analysis pipeline to surface the most meaningful visual moments.

**Parameters:**
- `url` *(required)* — YouTube video URL
- `max_frames` *(optional, default: `20`)* — Maximum frames to return (hard cap: `40`)
- `scene_threshold` *(optional, default: `0.25`)* — FFmpeg scene-detection sensitivity
- `output_width` *(optional, default: `640`)* — Width of returned JPEGs (proportional height)
- `min_importance_score` *(optional, default: `0.35`)* — Minimum composite quality score

**Returns:**
- Metadata: `video_id`, `duration_seconds`, `pipeline_stats`, `fallback_used`
- Frame blocks: each includes `timestamp_ms`, `composite_score`, and a `base64`-encoded JPEG image
- An index listing every returned frame with its timestamp and score

---

## The Frame Pipeline (5 Phases)

| Phase | Module | What It Does |
|-------|--------|--------------|
| **1. Download** | `downloader.py` | Downloads the video via `yt-dlp` with a duration guard and circuit-breaker for rate-limiting |
| **2. Extract** | `frame_extractor.py` | Detects scene changes with `ffmpeg` and extracts candidate frames |
| **3. Score** | `scorer.py` | Scores each frame across 5 signals: motion stability, entropy, edge density, rectangular coverage, and OCR word count |
| **4. Deduplicate** | `deduplicator.py` | Removes near-identical frames using DCT perceptual hashing (pHash) with a Hamming-distance threshold |
| **5. Load** | `frame_loader.py` | Resizes frames, compresses to JPEG (quality 85), and base64-encodes them for MCP transport |

---

## Quick Start

### Prerequisites

- **Python 3.11**
- **ffmpeg** — video processing
- **Tesseract OCR** — text detection in frames

**macOS:**
```bash
brew install ffmpeg tesseract
```

**Ubuntu / Linux:**
```bash
sudo apt-get install ffmpeg tesseract-ocr tesseract-ocr-eng
```

**Windows:** Download `ffmpeg` and Tesseract, then add both to your `PATH`.

Verify:
```bash
ffmpeg -version
tesseract --version
```

### Installation

```bash
# Clone the repository
git clone https://github.com/Ayush-Mamgain/youtube-mcp-server.git
cd youtube-mcp-server

# Create a virtual environment
python3 -m venv venv
source venv/bin/activate        # macOS / Linux
# venv\Scripts\activate      # Windows

# Install dependencies
pip install -r requirements.txt
```

### Configuration

Create a `.env` file in the project root:

```env
LOG_LEVEL=DEBUG
MCP_HTTP_PORT=8000
```

Optional tuning (loaded from `config.py` defaults if omitted):

| Variable | Default | Description |
|----------|---------|-------------|
| `MAX_VIDEO_DURATION` | `3600` | Reject videos longer than this (seconds) |
| `MAX_FRAMES_HARD_LIMIT` | `40` | Absolute cap on returned frames |
| `SCENE_THRESHOLD_DEFAULT` | `0.25` | FFmpeg scene-change threshold |
| `MIN_IMPORTANCE_SCORE_DEFAULT` | `0.35` | Minimum composite frame score |
| `OUTPUT_WIDTH_DEFAULT` | `640` | Width of returned JPEGs |

### Start the Server

```bash
python server.py
```

The server starts at `http://localhost:8000`.

---

## Connecting to Claude.ai (Local)

1. Start the server: `python server.py`
2. Go to **Claude.ai → Settings → Integrations → Add MCP server**
3. Enter: `http://localhost:8000/mcp`

For cloud-hosted Claude to reach your server, you'll need to expose it publicly (see Deployment below).

---

## API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `GET /healthz` | GET | Health check — verifies `ffmpeg`, `tesseract`, `yt-dlp`, and server readiness. Returns `{"status": "ok"}` or `{"status": "error", "detail": "..."}` |
| `POST /mcp` | POST | Streamable HTTP endpoint for MCP tool calls |

---

## Project Structure

```
youtube-mcp-server/
├── server.py              # MCP entry point — FastMCP + Starlette HTTP server
├── config.py              # Loads and validates environment variables
├── logger.py              # stderr-only logging
├── url_parser.py          # Validates YouTube URLs and extracts video IDs
├── transcript.py          # Fetches captions via youtube-transcript-api
├── downloader.py          # Phase 1 — video download with yt-dlp
├── frame_extractor.py     # Phase 2 — scene-change frame extraction via ffmpeg
├── scorer.py              # Phase 3 — multi-signal frame scoring (OpenCV + Tesseract)
├── deduplicator.py        # Phase 4 — perceptual-hash deduplication
├── frame_loader.py        # Phase 5 — resize, JPEG encode, base64
├── video_frames.py        # Orchestrates Phases 1–5 with semaphore and cleanup
├── requirements.txt       # Pinned Python dependencies
└── .gitignore             # Excludes .env, venv, caches, test artifacts
```

---

## Development Workflow

This project was built in **9 self-contained stages**, each with its own test file and verification step:

| Stage | Focus | Test File |
|-------|-------|-----------|
| 1 | Project scaffold, config, logger | `test_stage1.py` |
| 2 | YouTube URL parser & validation | `test_stage2.py` |
| 3 | Transcript fetching | `test_stage3.py` |
| 4 | Video downloader with duration guard | `test_stage4.py` |
| 5 | Frame extraction via ffmpeg | `test_stage5.py` |
| 6 | Multi-signal importance scoring | `test_stage6.py` |
| 7 | pHash deduplication + base64 loading | `test_stage7.py` |
| 8 | Full pipeline orchestration | `test_stage8.py` |
| 9 | MCP server entry point + health checks | `test_stage9.py` |

> **Golden rule:** Each stage is verified before proceeding. Run `python test_stage{N}.py` to validate.

---

## Deployment

> Docker support is planned but not yet configured. This section will be updated once containerization is complete.

For now, the server runs directly on any machine with Python 3.11, `ffmpeg`, and Tesseract installed. Recommended hosting options once Docker is ready:

- [Railway](https://railway.app)
- [Fly.io](https://fly.io)
- [Render](https://render.com)

Once deployed, update your Claude.ai integration URL to:
```
https://YOUR-DEPLOYMENT-URL/mcp
```

---

## Design Notes

- **All logging goes to stderr only.** `stdout` is reserved exclusively for MCP communication.
- **Thread-safe:** The `get_video_frames` pipeline uses a `threading.Semaphore(1)` to prevent concurrent downloads from overwhelming the system.
- **Automatic cleanup:** Temporary files created during video processing are deleted after each run.
- **Strict validation:** Video IDs are validated against `^[A-Za-z0-9_-]{11}$`. Playlist-only URLs, channel URLs, and malformed inputs are rejected with clear error messages.
- **Graceful degradation:** If all frames score below the minimum importance threshold, the pipeline falls back to the top 5 frames and flags `fallback_used: true`.

---

## License

MIT © [Ayush Mamgain](https://github.com/Ayush-Mamgain)

---

## Acknowledgments

Built with [FastMCP](https://github.com/jlowin/fastmcp), [Starlette](https://www.starlette.io), [yt-dlp](https://github.com/yt-dlp/yt-dlp), [youtube-transcript-api](https://github.com/jdepoix/youtube-transcript-api), and [OpenCV](https://opencv.org).