一个完全本地化的 MCP（模型上下文协议）服务器，为 AI 助手提供对 YouTube 视频的深度多模态感知能力。无需 API 密钥。 所有处理均通过 yt-dlp、OpenAI Whisper、FFmpeg、PySceneDetect 和 librosa 在本地运行。
注意： 本仓库还包含一个使用 Gemini API 的实验性 TypeScript 服务器 (
src/)。该服务器目前未处于活跃开发状态 —— Python 本地服务器 (server/) 是主要的实现版本。
Related MCP server: yt-analysis-mcp
YouTube URL
│
▼
yt-dlp ──────────────── download video.mp4
│ extract audio.wav (16 kHz mono)
▼
Whisper ─────────────── timestamped transcript (word-level)
│
▼
PySceneDetect ────────── detect scene-cut timestamps
│
▼
FFmpeg ──────────────── extract keyframe JPEGs at scene cuts
│
▼
OpenCV ──────────────── pixel-diff animation detection
│
▼
librosa ─────────────── energy · tempo · music vs speech
│
▼
timeline.py ─────────── unified JSON timeline (all signals, time-aligned)所有结果都缓存于 /tmp/yt-analysis-cache/<video_id>/ 中。再次调用相同的 URL 将实现即时响应。
# macOS
brew install ffmpeg
# Ubuntu / Debian
sudo apt install ffmpeg
# Verify
ffmpeg -version
python3 --version # must be 3.10+git clone https://github.com/yourusername/yt-mcp.git
cd yt-mcp
# Create and activate a virtual environment (recommended)
python3 -m venv .venv
source .venv/bin/activate # macOS / Linux
# .venv\Scripts\activate # Windows
pip install -r requirements.txtWhisper 模型权重会在首次转录调用时自动下载（base 模型约 75 MB，large 模型约 1.5 GB）。
MCP 客户端将服务器作为子进程启动 —— 它们不会自动激活您的 shell 或 venv。您必须使用绝对路径直接指向 venv 的 Python 解释器。
激活 venv 后查找您的解释器路径：
source .venv/bin/activate
which python # e.g. /Users/you/repos/yt-mcp/.venv/bin/pythonClaude Code：
claude mcp add -s user yt-mcp -- /path/to/yt-mcp/.venv/bin/python /path/to/yt-mcp/server/main.pyClaude Desktop — 添加到 ~/Library/Application Support/Claude/claude_desktop_config.json：
{
"mcpServers": {
"yt-mcp": {
"command": "/path/to/yt-mcp/.venv/bin/python",
"args": ["/path/to/yt-mcp/server/main.py"]
}
}
}将
/path/to/yt-mcp替换为您克隆仓库的绝对路径。 在 Windows 上，解释器位于.venv\Scripts\python.exe。
get_video_transcript使用 OpenAI Whisper 转录 YouTube 视频（完全在本地运行）。
参数 | 类型 | 默认值 | 描述 |
| string | — | 完整的 YouTube URL |
| string |
|
|
响应：
{
"title": "Video Title",
"duration": 847,
"language": "en",
"full_text": "Welcome to this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 4.5,
"text": "Welcome to this video.",
"words": [{ "word": "Welcome", "start": 0.0, "end": 0.6 }]
}
]
}get_video_frames提取关键帧作为 base64 编码的 JPEG。使用 PySceneDetect 进行场景检测，使用 FFmpeg 进行提取。
参数 | 类型 | 默认值 | 描述 |
| string | — | 完整的 YouTube URL |
| string |
|
|
| integer |
| 帧之间的秒数（用于 |
响应：
{
"title": "Video Title",
"duration": 847,
"duration_formatted": "14:07",
"frame_count": 12,
"strategy": "scene",
"frames": [
{
"t": 0.0,
"t_formatted": "0:00",
"keyframe": "<base64 JPEG>",
"scene_change": false,
"animation_detected": false
}
],
"summary": [ /* same list without keyframe bytes — for quick review */ ]
}get_audio_features使用 librosa 分析音频特征（在本地运行）。
参数 | 类型 | 默认值 | 描述 |
| string | — | 完整的 YouTube URL |
| integer |
| 分析窗口大小（秒） |
响应：
{
"title": "Video Title",
"duration": 847,
"segment_duration": 30,
"segments": [
{
"t_start": 0.0,
"t_end": 30.0,
"energy": "medium",
"music": false,
"tempo_bpm": 95.0,
"rms_db": -22.1
}
]
}get_full_context核心工具。 返回一个完整、同步的多模态时间轴 —— 转录内容 + 场景边界 + 动画检测 + 音频特征，所有内容均已按时间对齐。
参数 | 类型 | 默认值 | 描述 |
| string | — | 完整的 YouTube URL |
| boolean |
| 在每个片段中嵌入 base64 关键帧 |
| string |
| Whisper 模型大小 |
响应：
{
"title": "How Transformers Work",
"channel": "AI Explained",
"duration": 847,
"duration_formatted": "14:07",
"language": "en",
"description": "In this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 12.0,
"transcript": "Welcome to this video on transformers...",
"keyframe": null,
"scene_change": false,
"animation_detected": false,
"audio": {
"energy": "low",
"speech_rate": "normal",
"music": true,
"tempo_bpm": 0.0,
"rms_db": -28.4
}
}
]
}上下文窗口提示： 先调用
get_full_context并设置include_frames=false以了解视频结构，然后针对感兴趣的特定时间戳调用get_video_frames。
https://www.youtube.com/watch?v=VIDEO_ID
https://youtu.be/VIDEO_ID
https://youtube.com/shorts/VIDEO_ID变量 | 默认值 | 描述 |
|
| 用于下载视频和音频的缓存目录 |
# Activate the venv first
source .venv/bin/activate
# Run the server directly (stdio mode — same as MCP clients use)
python server/main.py
# Quick smoke test
python -c "
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
d = VideoDownloader()
vp, ap, info = d.download('https://www.youtube.com/watch?v=jNQXAC9IVRw')
print(get_transcript(ap)['language'])
"Python 服务器拥有完整的单元测试套件 —— 涵盖 6 个模块的 164 个测试。所有测试均在没有网络访问或模型下载的情况下运行；每个外部依赖项（Whisper、librosa、FFmpeg、PySceneDetect、OpenCV、yt-dlp）均已模拟。
pip install -r requirements-dev.txtpython -m pytest预期输出：164 passed in ~4s
python -m pytest tests/test_downloader.py # VideoDownloader + VideoInfo
python -m pytest tests/test_transcript.py # Whisper wrapper + range helpers
python -m pytest tests/test_frames.py # FFmpeg, PySceneDetect, OpenCV
python -m pytest tests/test_audio.py # librosa AudioAnalyzer
python -m pytest tests/test_timeline.py # build_timeline + speech rate
python -m pytest tests/test_main.py # all 4 MCP tool handlerspython -m pytest tests/test_timeline.py::TestBuildTimeline::test_rapid_cuts_below_min_merged -v下面的示例使用了 プリマドンナ / 星街すいせい (星街彗星 · Suisei Channel, 2:52) —— 这是一个日本音乐视频，它测试了管道的每一层：多语言 Whisper 转录、通过 librosa HPSS 进行音乐检测、通过 PySceneDetect 进行快速场景切换，以及通过 OpenCV 像素差异进行动画检测。
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
from server.tools.audio import AudioAnalyzer
from server.tools.frames import detect_scene_timestamps
URL = "https://www.youtube.com/watch?v=M1GYqy0tHV0"
d = VideoDownloader()
video_path, audio_path, info = d.download(URL)
print(f"Title: {info.title}") # プリマドンナ / 星街すいせい(official)
print(f"Duration: {info.duration:.0f}s") # 172
transcript = get_transcript(audio_path, model_size="base")
print(f"Language: {transcript['language']}") # ja
cuts = detect_scene_timestamps(video_path)
print(f"Scene cuts detected: {len(cuts)}") # typically 30–60 for a music video
analyzer = AudioAnalyzer(audio_path)
seg = analyzer.analyze_segment(0, 30)
print(f"First 30s — energy: {seg['energy']}, music: {seg['music']}")
# energy: 'medium' or 'high', music: True有关完整的测试指南（固定装置、模拟模式、为新工具编写测试），请参阅 docs/testing.md。
有关系统设计、数据流以及如何添加新工具的详细说明：
docs/architecture.md — 管道图和关键设计决策
docs/python-server.md — 所有模块的组件参考
docs/extending.md — 如何添加新工具
docs/testing.md — 测试套件结构、固定装置和编写新测试
src/ 目录包含一个实验性的 TypeScript 服务器，它将视频分析委托给 Gemini API。它目前未处于活跃开发状态，仅供参考。
如果您正在寻找基于云的快速视频问答，TypeScript 服务器的方法（直接将 YouTube URL 传递给 Gemini）非常适合快速原型设计 —— 但 Python 服务器是唯一将获得持续维护的实现。
请参阅 docs/typescript-server.md 获取其 API 参考。
Ask a specific question about a YouTube video's content. Returns an answer based on the video.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| question | Yes | Your question about the video content |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden but only states it 'Returns an answer based on the video'. It does not disclose any behavioral traits such as rate limits, authentication needs, or what type of questions are supported. The statement is minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core purpose efficiently. No wasted words, though it could include more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 parameters and no output schema, the description provides a minimal but functional explanation. It lacks details about the format of the answer or limitations, which might be needed for effective use. However, it is not incomplete to the point of being unusable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters adequately. The description adds no new meaning beyond the schema, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Describes the tool as asking a specific question about a YouTube video and returning an answer. The verb 'ask' and resource 'video content' are clear. However, it does not explicitly differentiate from sibling tools like 'summarize_video', which also deals with video content, though the 'specific question' aspect provides some distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when needing to ask a question about video content, but does not provide explicit guidance on when to use this tool versus alternatives like 'summarize_video' or when not to use it. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract frames from a YouTube video at specific timestamps you provide. Use this when you already know the exact timestamps you want (e.g., from get_video_timestamps or video summary).
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| timestamps | Yes | Array of timestamps in seconds to extract frames from (e.g., [5, 30, 60, 120]) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It focuses on the extraction function but doesn't detail behavior like file output format, naming conventions, or error handling. The description adds some context beyond schema but is not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, directly stating the purpose and usage context with no fluff. It is concise and front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (4 parameters with 2 required) and no output schema, the description is adequate but not exhaustive. It covers the primary use case but lacks details on output behavior, which is acceptable since there is no output schema to complement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by clarifying the usage context (timestamps from other tools) but does not provide additional semantic meaning beyond what the schema already offers for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts frames from a YouTube video at specific timestamps. It specifies the resource (YouTube video) and the action (extract frames at timestamps), and distinguishes it from siblings like extract_screenshots by focusing on user-provided timestamps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: when you already know exact timestamps. It suggests examples like using timestamps from get_video_timestamps or video summary, implying when not to use it (if you don't have timestamps) and providing context for alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract key screenshots from a YouTube video at important moments. Uses AI to identify visually significant timestamps, then extracts frames. Returns both base64 images and optionally saves to disk.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of screenshots to extract (1-20, default: 5) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it uses AI for timestamp selection, returns base64 images, and optionally saves to disk. With no annotations provided, this description carries the burden well. However, more details about performance or API calls could enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each providing distinct value: purpose, method, and output. Could be more concise by removing 'Optionally saves to disk' since it's covered in output_dir parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main aspects: video input, AI selection, number of screenshots, output format, and save behavior. However, no output schema means description could mention return format (array of base64 strings). Overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds context for 'focus' parameter but doesn't elaborate on 'resolution' enum meaning beyond schema. The 'output_dir' description adds context about fallback behavior, which adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it extracts screenshots from YouTube videos at important moments using AI. Distinguishes from siblings like 'extract_frames' by mentioning AI to find visually significant timestamps, but could be more specific about difference from 'extract_frames'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for extracting key screenshots, but no explicit guidance on when to use vs siblings like 'extract_frames' or 'get_video_timestamps'. No mention of prerequisites (e.g., need ffmpeg).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Preview mode: Use AI to identify important moments in a YouTube video and return their timestamps WITHOUT extracting frames. Use this to preview what timestamps would be selected before committing to extraction.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of timestamps to identify (1-20, default: 5) | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states that it uses AI to identify timestamps and does not extract frames. However, it does not disclose other behavioral traits such as API rate limits, how the AI works, or constraints on video length/format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences. The first sentence clearly defines the tool's function, and the second sentence explains its purpose relative to other tools. No unnecessary information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is fairly complete. It explains the tool's purpose and its relationship to extraction tools. However, it does not mention return format or behavior for invalid URLs, but the schema provides validation hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides descriptions for all parameters (100% coverage), so the description adds minimal value beyond the schema. However, the description does give context for the 'focus' parameter by providing examples like 'product demos' and 'code examples', which adds nuance beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to use AI to identify important moments in a YouTube video and return their timestamps without extracting frames. It distinguishes itself from sibling tools by emphasizing that it is a preview mode that does not extract frames, which sets it apart from extract_frames and extract_screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'to preview what timestamps would be selected before committing to extraction.' However, it does not explicitly mention when not to use it or suggest alternatives, but the sibling list makes it clear that extraction tools exist for different purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize a YouTube video's content. Returns a text summary based on the specified detail level.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| detail_level | No | Level of detail: brief (2-3 sentences), medium (key points with timestamps), detailed (comprehensive breakdown) | medium |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral aspects. It mentions it returns a text summary based on detail level, but does not disclose processing time, input validation, or error handling. The behavior is straightforward but minimally described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first states the core function, second explains output based on parameter. It is concise but could be more front-loaded about what the user gets. No fluff, but slightly vague on output format.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain return format. It says 'returns a text summary' but not whether it's plain text, structured, or includes timestamps. With siblings like ask_about_video, more detail on output would help. Scores 3 as it covers basic purpose but lacks output specificity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the detail_level enum values (brief, medium, detailed) with concrete expectations like 'key points with timestamps', which is not in the schema. This extra context earns a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it summarizes a YouTube video's content and returns a text summary based on a detail level. It distinguishes from siblings like ask_about_video (querying) and extract_frames (visual extraction), but could be more explicit about the summary format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives are provided. The description implies usage when a text summary is needed, but does not clarify when to choose this over ask_about_video or other tools. However, the enum options for detail_level give some usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0ask_about_videoextract_framesextract_screenshotsget_video_timestampssummarize_videoScored across 5 tools
Tools are mostly distinct: ask_about_video and summarize_video both deal with content analysis but differ in question vs. summary. extract_frames and extract_screenshots have clear differences (given timestamps vs. AI-chosen moments). get_video_timestamps is a preview for extraction, reducing overlap. Minor confusion possible between ask_about_video and summarize_video.
All tool names use verb_* pattern with snake_case (ask_about_video, extract_frames, etc.), which is consistent. Some verbs are compound (ask_about), but the pattern is predictable and clear.
5 tools is appropriate for a YouTube MCP: they cover core operations (summarize, Q&A, frame extraction, screenshot extraction, timestamp preview). Not too few or too many.
The tool set covers main user needs: summarization, Q&A, and visual extraction with a preview step. Missing operations like searching videos or managing playlists, but those may be out of scope. The workflow from preview to extraction is well-supported.
Gemini watches public YouTube videos and returns a timestamped brief or answers to your questions.
YouTube transcripts, video search, chapters and extractive summaries. No YouTube API key, no LLM.
Summarize YouTube videos or fetch transcripts. Pay-per-call via x402 (USDC on Base), no account.
Transcribe YouTube via Whisper. Summaries, chapters, semantic-search across your corpus.
