Полностью локальный сервер MCP (Model Context Protocol), который дает ИИ-ассистентам глубокое мультимодальное понимание видео на YouTube. API-ключи не требуются. Вся обработка выполняется на устройстве с помощью yt-dlp, OpenAI Whisper, FFmpeg, PySceneDetect и librosa.
Примечание: Этот репозиторий также содержит экспериментальный сервер на TypeScript (
src/), использующий API Gemini. Этот сервер не находится в активной разработке — основной реализацией является локальный сервер на Python (server/).
Related MCP server: yt-analysis-mcp
YouTube URL
│
▼
yt-dlp ──────────────── download video.mp4
│ extract audio.wav (16 kHz mono)
▼
Whisper ─────────────── timestamped transcript (word-level)
│
▼
PySceneDetect ────────── detect scene-cut timestamps
│
▼
FFmpeg ──────────────── extract keyframe JPEGs at scene cuts
│
▼
OpenCV ──────────────── pixel-diff animation detection
│
▼
librosa ─────────────── energy · tempo · music vs speech
│
▼
timeline.py ─────────── unified JSON timeline (all signals, time-aligned)Все результаты кэшируются в /tmp/yt-analysis-cache/<video_id>/. Повторный вызов того же URL происходит мгновенно.
# macOS
brew install ffmpeg
# Ubuntu / Debian
sudo apt install ffmpeg
# Verify
ffmpeg -version
python3 --version # must be 3.10+git clone https://github.com/yourusername/yt-mcp.git
cd yt-mcp
# Create and activate a virtual environment (recommended)
python3 -m venv .venv
source .venv/bin/activate # macOS / Linux
# .venv\Scripts\activate # Windows
pip install -r requirements.txtВеса модели Whisper загружаются автоматически при первом вызове транскрибации (~75 МБ для base, ~1.5 ГБ для large).
Клиенты MCP запускают сервер как подпроцесс — они не активируют вашу оболочку или venv автоматически. Вы должны указать им путь напрямую к интерпретатору Python внутри venv, используя его абсолютный путь.
Найдите путь к вашему интерпретатору после активации venv:
source .venv/bin/activate
which python # e.g. /Users/you/repos/yt-mcp/.venv/bin/pythonClaude Code:
claude mcp add -s user yt-mcp -- /path/to/yt-mcp/.venv/bin/python /path/to/yt-mcp/server/main.pyClaude Desktop — добавьте в ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"yt-mcp": {
"command": "/path/to/yt-mcp/.venv/bin/python",
"args": ["/path/to/yt-mcp/server/main.py"]
}
}
}Замените
/path/to/yt-mcpна абсолютный путь к месту, куда вы клонировали репозиторий. В Windows интерпретатор находится по адресу.venv\Scripts\python.exe.
get_video_transcriptТранскрибирует видео с YouTube с помощью OpenAI Whisper (работает полностью локально).
Параметр | Тип | По умолчанию | Описание |
| string | — | Полный URL видео YouTube |
| string |
|
|
Ответ:
{
"title": "Video Title",
"duration": 847,
"language": "en",
"full_text": "Welcome to this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 4.5,
"text": "Welcome to this video.",
"words": [{ "word": "Welcome", "start": 0.0, "end": 0.6 }]
}
]
}get_video_framesИзвлекает ключевые кадры в виде JPEG, закодированных в base64. Использует PySceneDetect для обнаружения сцен и FFmpeg для извлечения.
Параметр | Тип | По умолчанию | Описание |
| string | — | Полный URL видео YouTube |
| string |
|
|
| integer |
| Секунды между кадрами (для стратегий |
Ответ:
{
"title": "Video Title",
"duration": 847,
"duration_formatted": "14:07",
"frame_count": 12,
"strategy": "scene",
"frames": [
{
"t": 0.0,
"t_formatted": "0:00",
"keyframe": "<base64 JPEG>",
"scene_change": false,
"animation_detected": false
}
],
"summary": [ /* same list without keyframe bytes — for quick review */ ]
}get_audio_featuresАнализирует характеристики аудио с помощью librosa (работает локально).
Параметр | Тип | По умолчанию | Описание |
| string | — | Полный URL видео YouTube |
| integer |
| Размер окна анализа в секундах |
Ответ:
{
"title": "Video Title",
"duration": 847,
"segment_duration": 30,
"segments": [
{
"t_start": 0.0,
"t_end": 30.0,
"energy": "medium",
"music": false,
"tempo_bpm": 95.0,
"rms_db": -22.1
}
]
}get_full_contextОсновной инструмент. Возвращает полный синхронизированный мультимодальный таймлайн — транскрипт + границы сцен + обнаружение анимации + аудиохарактеристики, все выровнено по времени.
Параметр | Тип | По умолчанию | Описание |
| string | — | Полный URL видео YouTube |
| boolean |
| Внедрить ключевые кадры base64 для каждого сегмента |
| string |
| Размер модели Whisper |
Ответ:
{
"title": "How Transformers Work",
"channel": "AI Explained",
"duration": 847,
"duration_formatted": "14:07",
"language": "en",
"description": "In this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 12.0,
"transcript": "Welcome to this video on transformers...",
"keyframe": null,
"scene_change": false,
"animation_detected": false,
"audio": {
"energy": "low",
"speech_rate": "normal",
"music": true,
"tempo_bpm": 0.0,
"rms_db": -28.4
}
}
]
}Совет по контекстному окну: Сначала вызовите
get_full_contextсinclude_frames=false, чтобы понять структуру видео, а затем вызовитеget_video_framesдля конкретных интересующих вас временных меток.
https://www.youtube.com/watch?v=VIDEO_ID
https://youtu.be/VIDEO_ID
https://youtube.com/shorts/VIDEO_IDПеременная | По умолчанию | Описание |
|
| Директория кэша для загруженных видео и аудио |
# Activate the venv first
source .venv/bin/activate
# Run the server directly (stdio mode — same as MCP clients use)
python server/main.py
# Quick smoke test
python -c "
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
d = VideoDownloader()
vp, ap, info = d.download('https://www.youtube.com/watch?v=jNQXAC9IVRw')
print(get_transcript(ap)['language'])
"Python-сервер имеет полный набор модульных тестов — 164 теста в 6 модулях. Все тесты выполняются без доступа к сети или загрузки моделей; каждая внешняя зависимость (Whisper, librosa, FFmpeg, PySceneDetect, OpenCV, yt-dlp) имитируется (mocked).
pip install -r requirements-dev.txtpython -m pytestОжидаемый вывод: 164 passed in ~4s
python -m pytest tests/test_downloader.py # VideoDownloader + VideoInfo
python -m pytest tests/test_transcript.py # Whisper wrapper + range helpers
python -m pytest tests/test_frames.py # FFmpeg, PySceneDetect, OpenCV
python -m pytest tests/test_audio.py # librosa AudioAnalyzer
python -m pytest tests/test_timeline.py # build_timeline + speech rate
python -m pytest tests/test_main.py # all 4 MCP tool handlerspython -m pytest tests/test_timeline.py::TestBuildTimeline::test_rapid_cuts_below_min_merged -vВ примере ниже используется プリマドンナ / 星街すいせい (Hoshimachi Suisei · Suisei Channel, 2:52) — японский музыкальный клип, который задействует каждый уровень конвейера: многоязычная транскрипция Whisper, обнаружение музыки через librosa HPSS, быстрые склейки сцен через PySceneDetect и обнаружение анимации через разницу пикселей OpenCV.
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
from server.tools.audio import AudioAnalyzer
from server.tools.frames import detect_scene_timestamps
URL = "https://www.youtube.com/watch?v=M1GYqy0tHV0"
d = VideoDownloader()
video_path, audio_path, info = d.download(URL)
print(f"Title: {info.title}") # プリマドンナ / 星街すいせい(official)
print(f"Duration: {info.duration:.0f}s") # 172
transcript = get_transcript(audio_path, model_size="base")
print(f"Language: {transcript['language']}") # ja
cuts = detect_scene_timestamps(video_path)
print(f"Scene cuts detected: {len(cuts)}") # typically 30–60 for a music video
analyzer = AudioAnalyzer(audio_path)
seg = analyzer.analyze_segment(0, 30)
print(f"First 30s — energy: {seg['energy']}, music: {seg['music']}")
# energy: 'medium' or 'high', music: TrueПолное руководство по тестированию — фикстуры, шаблоны имитации, написание тестов для новых инструментов — см. в docs/testing.md.
Для подробного объяснения проектирования системы, потоков данных и того, как добавлять новые инструменты:
docs/architecture.md — диаграммы конвейера и ключевые проектные решения
docs/python-server.md — справочник компонентов для всех модулей
docs/extending.md — как добавлять новые инструменты
docs/testing.md — структура набора тестов, фикстуры и написание новых тестов
Директория src/ содержит экспериментальный сервер на TypeScript, который делегирует анализ видео API Gemini. Он не находится в активной разработке и сохранен только для справки.
Если вы ищете быстрые облачные ответы на вопросы по видео, подход сервера на TypeScript (передача URL YouTube напрямую в Gemini) хорошо работает для быстрого прототипа, но сервер на Python — это единственная реализация, которая будет получать постоянную поддержку.
См. docs/typescript-server.md для получения справочной информации по его API.
Ask a specific question about a YouTube video's content. Returns an answer based on the video.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| question | Yes | Your question about the video content |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden but only states it 'Returns an answer based on the video'. It does not disclose any behavioral traits such as rate limits, authentication needs, or what type of questions are supported. The statement is minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core purpose efficiently. No wasted words, though it could include more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 parameters and no output schema, the description provides a minimal but functional explanation. It lacks details about the format of the answer or limitations, which might be needed for effective use. However, it is not incomplete to the point of being unusable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters adequately. The description adds no new meaning beyond the schema, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Describes the tool as asking a specific question about a YouTube video and returning an answer. The verb 'ask' and resource 'video content' are clear. However, it does not explicitly differentiate from sibling tools like 'summarize_video', which also deals with video content, though the 'specific question' aspect provides some distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when needing to ask a question about video content, but does not provide explicit guidance on when to use this tool versus alternatives like 'summarize_video' or when not to use it. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract frames from a YouTube video at specific timestamps you provide. Use this when you already know the exact timestamps you want (e.g., from get_video_timestamps or video summary).
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| timestamps | Yes | Array of timestamps in seconds to extract frames from (e.g., [5, 30, 60, 120]) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It focuses on the extraction function but doesn't detail behavior like file output format, naming conventions, or error handling. The description adds some context beyond schema but is not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, directly stating the purpose and usage context with no fluff. It is concise and front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (4 parameters with 2 required) and no output schema, the description is adequate but not exhaustive. It covers the primary use case but lacks details on output behavior, which is acceptable since there is no output schema to complement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by clarifying the usage context (timestamps from other tools) but does not provide additional semantic meaning beyond what the schema already offers for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts frames from a YouTube video at specific timestamps. It specifies the resource (YouTube video) and the action (extract frames at timestamps), and distinguishes it from siblings like extract_screenshots by focusing on user-provided timestamps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: when you already know exact timestamps. It suggests examples like using timestamps from get_video_timestamps or video summary, implying when not to use it (if you don't have timestamps) and providing context for alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract key screenshots from a YouTube video at important moments. Uses AI to identify visually significant timestamps, then extracts frames. Returns both base64 images and optionally saves to disk.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of screenshots to extract (1-20, default: 5) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it uses AI for timestamp selection, returns base64 images, and optionally saves to disk. With no annotations provided, this description carries the burden well. However, more details about performance or API calls could enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each providing distinct value: purpose, method, and output. Could be more concise by removing 'Optionally saves to disk' since it's covered in output_dir parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main aspects: video input, AI selection, number of screenshots, output format, and save behavior. However, no output schema means description could mention return format (array of base64 strings). Overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds context for 'focus' parameter but doesn't elaborate on 'resolution' enum meaning beyond schema. The 'output_dir' description adds context about fallback behavior, which adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it extracts screenshots from YouTube videos at important moments using AI. Distinguishes from siblings like 'extract_frames' by mentioning AI to find visually significant timestamps, but could be more specific about difference from 'extract_frames'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for extracting key screenshots, but no explicit guidance on when to use vs siblings like 'extract_frames' or 'get_video_timestamps'. No mention of prerequisites (e.g., need ffmpeg).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Preview mode: Use AI to identify important moments in a YouTube video and return their timestamps WITHOUT extracting frames. Use this to preview what timestamps would be selected before committing to extraction.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of timestamps to identify (1-20, default: 5) | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states that it uses AI to identify timestamps and does not extract frames. However, it does not disclose other behavioral traits such as API rate limits, how the AI works, or constraints on video length/format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences. The first sentence clearly defines the tool's function, and the second sentence explains its purpose relative to other tools. No unnecessary information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is fairly complete. It explains the tool's purpose and its relationship to extraction tools. However, it does not mention return format or behavior for invalid URLs, but the schema provides validation hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides descriptions for all parameters (100% coverage), so the description adds minimal value beyond the schema. However, the description does give context for the 'focus' parameter by providing examples like 'product demos' and 'code examples', which adds nuance beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to use AI to identify important moments in a YouTube video and return their timestamps without extracting frames. It distinguishes itself from sibling tools by emphasizing that it is a preview mode that does not extract frames, which sets it apart from extract_frames and extract_screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'to preview what timestamps would be selected before committing to extraction.' However, it does not explicitly mention when not to use it or suggest alternatives, but the sibling list makes it clear that extraction tools exist for different purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize a YouTube video's content. Returns a text summary based on the specified detail level.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| detail_level | No | Level of detail: brief (2-3 sentences), medium (key points with timestamps), detailed (comprehensive breakdown) | medium |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral aspects. It mentions it returns a text summary based on detail level, but does not disclose processing time, input validation, or error handling. The behavior is straightforward but minimally described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first states the core function, second explains output based on parameter. It is concise but could be more front-loaded about what the user gets. No fluff, but slightly vague on output format.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain return format. It says 'returns a text summary' but not whether it's plain text, structured, or includes timestamps. With siblings like ask_about_video, more detail on output would help. Scores 3 as it covers basic purpose but lacks output specificity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the detail_level enum values (brief, medium, detailed) with concrete expectations like 'key points with timestamps', which is not in the schema. This extra context earns a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it summarizes a YouTube video's content and returns a text summary based on a detail level. It distinguishes from siblings like ask_about_video (querying) and extract_frames (visual extraction), but could be more explicit about the summary format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives are provided. The description implies usage when a text summary is needed, but does not clarify when to choose this over ask_about_video or other tools. However, the enum options for detail_level give some usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0ask_about_videoextract_framesextract_screenshotsget_video_timestampssummarize_videoScored across 5 tools
Tools are mostly distinct: ask_about_video and summarize_video both deal with content analysis but differ in question vs. summary. extract_frames and extract_screenshots have clear differences (given timestamps vs. AI-chosen moments). get_video_timestamps is a preview for extraction, reducing overlap. Minor confusion possible between ask_about_video and summarize_video.
All tool names use verb_* pattern with snake_case (ask_about_video, extract_frames, etc.), which is consistent. Some verbs are compound (ask_about), but the pattern is predictable and clear.
5 tools is appropriate for a YouTube MCP: they cover core operations (summarize, Q&A, frame extraction, screenshot extraction, timestamp preview). Not too few or too many.
The tool set covers main user needs: summarization, Q&A, and visual extraction with a preview step. Missing operations like searching videos or managing playlists, but those may be out of scope. The workflow from preview to extraction is well-supported.
Gemini watches public YouTube videos and returns a timestamped brief or answers to your questions.
YouTube transcripts, video search, chapters and extractive summaries. No YouTube API key, no LLM.
Summarize YouTube videos or fetch transcripts. Pay-per-call via x402 (USDC on Base), no account.
Transcribe YouTube via Whisper. Summaries, chapters, semantic-search across your corpus.
