AIアシスタントにYouTube動画の深いマルチモーダル認識能力を与える、完全にローカルで動作するMCP（Model Context Protocol）サーバーです。APIキーは不要です。 すべての処理は、yt-dlp、OpenAI Whisper、FFmpeg、PySceneDetect、librosaを使用してデバイス上で実行されます。
注: このリポジトリには、Gemini APIを使用する実験的なTypeScriptサーバー（
src/）も含まれています。そのサーバーは現在開発が行われておらず、Pythonローカルサーバー（server/）が主要な実装となります。
Related MCP server: yt-analysis-mcp
YouTube URL
│
▼
yt-dlp ──────────────── download video.mp4
│ extract audio.wav (16 kHz mono)
▼
Whisper ─────────────── timestamped transcript (word-level)
│
▼
PySceneDetect ────────── detect scene-cut timestamps
│
▼
FFmpeg ──────────────── extract keyframe JPEGs at scene cuts
│
▼
OpenCV ──────────────── pixel-diff animation detection
│
▼
librosa ─────────────── energy · tempo · music vs speech
│
▼
timeline.py ─────────── unified JSON timeline (all signals, time-aligned)すべての結果は /tmp/yt-analysis-cache/<video_id>/ にキャッシュされます。同じURLを再度呼び出すと即座に結果が返されます。
# macOS
brew install ffmpeg
# Ubuntu / Debian
sudo apt install ffmpeg
# Verify
ffmpeg -version
python3 --version # must be 3.10+git clone https://github.com/yourusername/yt-mcp.git
cd yt-mcp
# Create and activate a virtual environment (recommended)
python3 -m venv .venv
source .venv/bin/activate # macOS / Linux
# .venv\Scripts\activate # Windows
pip install -r requirements.txtWhisperモデルの重みは、最初の文字起こし呼び出し時に自動的にダウンロードされます（baseで約75MB、largeで約1.5GB）。
MCPクライアントはサーバーをサブプロセスとして起動するため、シェルやvenvを自動的にアクティブ化することはありません。venvのPythonインタープリターの絶対パスを直接指定する必要があります。
venvをアクティブ化した後、インタープリターのパスを確認してください：
source .venv/bin/activate
which python # e.g. /Users/you/repos/yt-mcp/.venv/bin/pythonClaude Code:
claude mcp add -s user yt-mcp -- /path/to/yt-mcp/.venv/bin/python /path/to/yt-mcp/server/main.pyClaude Desktop — ~/Library/Application Support/Claude/claude_desktop_config.json に追加します：
{
"mcpServers": {
"yt-mcp": {
"command": "/path/to/yt-mcp/.venv/bin/python",
"args": ["/path/to/yt-mcp/server/main.py"]
}
}
}
/path/to/yt-mcpをリポジトリをクローンした場所の絶対パスに置き換えてください。 Windowsの場合、インタープリターは.venv\Scripts\python.exeにあります。
get_video_transcriptOpenAI Whisperを使用してYouTube動画を文字起こしします（完全にローカルで実行）。
パラメータ | 型 | デフォルト | 説明 |
| string | — | YouTubeの完全なURL |
| string |
|
|
レスポンス:
{
"title": "Video Title",
"duration": 847,
"language": "en",
"full_text": "Welcome to this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 4.5,
"text": "Welcome to this video.",
"words": [{ "word": "Welcome", "start": 0.0, "end": 0.6 }]
}
]
}get_video_framesキーフレームをbase64エンコードされたJPEGとして抽出します。シーン検出にはPySceneDetectを、抽出にはFFmpegを使用します。
パラメータ | 型 | デフォルト | 説明 |
| string | — | YouTubeの完全なURL |
| string |
|
|
| integer |
| フレーム間の秒数（ |
レスポンス:
{
"title": "Video Title",
"duration": 847,
"duration_formatted": "14:07",
"frame_count": 12,
"strategy": "scene",
"frames": [
{
"t": 0.0,
"t_formatted": "0:00",
"keyframe": "<base64 JPEG>",
"scene_change": false,
"animation_detected": false
}
],
"summary": [ /* same list without keyframe bytes — for quick review */ ]
}get_audio_featureslibrosaを使用してオーディオ特性を分析します（ローカルで実行）。
パラメータ | 型 | デフォルト | 説明 |
| string | — | YouTubeの完全なURL |
| integer |
| 分析ウィンドウサイズ（秒） |
レスポンス:
{
"title": "Video Title",
"duration": 847,
"segment_duration": 30,
"segments": [
{
"t_start": 0.0,
"t_end": 30.0,
"energy": "medium",
"music": false,
"tempo_bpm": 95.0,
"rms_db": -22.1
}
]
}get_full_context主要ツール。 完全で同期されたマルチモーダルタイムライン（文字起こし + シーン境界 + アニメーション検出 + オーディオ特性）をすべて時間軸に合わせて返します。
パラメータ | 型 | デフォルト | 説明 |
| string | — | YouTubeの完全なURL |
| boolean |
| セグメントごとにbase64キーフレームを埋め込む |
| string |
| Whisperモデルサイズ |
レスポンス:
{
"title": "How Transformers Work",
"channel": "AI Explained",
"duration": 847,
"duration_formatted": "14:07",
"language": "en",
"description": "In this video...",
"segments": [
{
"t_start": 0.0,
"t_end": 12.0,
"transcript": "Welcome to this video on transformers...",
"keyframe": null,
"scene_change": false,
"animation_detected": false,
"audio": {
"energy": "low",
"speech_rate": "normal",
"music": true,
"tempo_bpm": 0.0,
"rms_db": -28.4
}
}
]
}コンテキストウィンドウのヒント: まず
include_frames=falseでget_full_contextを呼び出して動画の構造を把握し、その後、特定のタイムスタンプについてget_video_framesを呼び出してください。
https://www.youtube.com/watch?v=VIDEO_ID
https://youtu.be/VIDEO_ID
https://youtube.com/shorts/VIDEO_ID変数 | デフォルト | 説明 |
|
| ダウンロードした動画やオーディオのキャッシュディレクトリ |
# Activate the venv first
source .venv/bin/activate
# Run the server directly (stdio mode — same as MCP clients use)
python server/main.py
# Quick smoke test
python -c "
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
d = VideoDownloader()
vp, ap, info = d.download('https://www.youtube.com/watch?v=jNQXAC9IVRw')
print(get_transcript(ap)['language'])
"Pythonサーバーには、6つのモジュールにわたる164のテストを含む完全なユニットテストスイートがあります。すべてのテストはネットワークアクセスやモデルのダウンロードなしで実行されます。すべての外部依存関係（Whisper、librosa、FFmpeg、PySceneDetect、OpenCV、yt-dlp）はモック化されています。
pip install -r requirements-dev.txtpython -m pytest期待される出力: 164 passed in ~4s
python -m pytest tests/test_downloader.py # VideoDownloader + VideoInfo
python -m pytest tests/test_transcript.py # Whisper wrapper + range helpers
python -m pytest tests/test_frames.py # FFmpeg, PySceneDetect, OpenCV
python -m pytest tests/test_audio.py # librosa AudioAnalyzer
python -m pytest tests/test_timeline.py # build_timeline + speech rate
python -m pytest tests/test_main.py # all 4 MCP tool handlerspython -m pytest tests/test_timeline.py::TestBuildTimeline::test_rapid_cuts_below_min_merged -v以下の例では、プリマドンナ / 星街すいせい（星街すいせい · Suisei Channel, 2:52）を使用しています。これは、多言語Whisper文字起こし、librosa HPSSによる音楽検出、PySceneDetectによる高速シーンカット、OpenCVピクセル差分によるアニメーション検出など、パイプラインのあらゆる層を検証する日本のミュージックビデオです。
from server.utils.downloader import VideoDownloader
from server.tools.transcript import get_transcript
from server.tools.audio import AudioAnalyzer
from server.tools.frames import detect_scene_timestamps
URL = "https://www.youtube.com/watch?v=M1GYqy0tHV0"
d = VideoDownloader()
video_path, audio_path, info = d.download(URL)
print(f"Title: {info.title}") # プリマドンナ / 星街すいせい(official)
print(f"Duration: {info.duration:.0f}s") # 172
transcript = get_transcript(audio_path, model_size="base")
print(f"Language: {transcript['language']}") # ja
cuts = detect_scene_timestamps(video_path)
print(f"Scene cuts detected: {len(cuts)}") # typically 30–60 for a music video
analyzer = AudioAnalyzer(audio_path)
seg = analyzer.analyze_segment(0, 30)
print(f"First 30s — energy: {seg['energy']}, music: {seg['music']}")
# energy: 'medium' or 'high', music: Trueフィクスチャ、モックパターン、新しいツールのテスト作成など、テストガイドの詳細は docs/testing.md を参照してください。
システム設計、データフロー、新しいツールの追加方法の詳細については、以下を参照してください：
docs/architecture.md — パイプライン図と主要な設計決定
docs/python-server.md — 全モジュールのコンポーネントリファレンス
docs/extending.md — 新しいツールの追加方法
docs/testing.md — テストスイートの構造、フィクスチャ、新しいテストの作成方法
src/ ディレクトリには、動画分析をGemini APIに委譲する実験的なTypeScriptサーバーが含まれています。これは現在開発が行われておらず、参照用としてのみ保持されています。
高速なクラウドベースの動画Q&Aをお探しの場合は、TypeScriptサーバーのアプローチ（YouTube URLを直接Geminiに渡す）が簡単なプロトタイプには適していますが、継続的なメンテナンスが行われるのはPythonサーバーのみです。
APIリファレンスについては docs/typescript-server.md を参照してください。
Ask a specific question about a YouTube video's content. Returns an answer based on the video.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| question | Yes | Your question about the video content |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden but only states it 'Returns an answer based on the video'. It does not disclose any behavioral traits such as rate limits, authentication needs, or what type of questions are supported. The statement is minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core purpose efficiently. No wasted words, though it could include more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 parameters and no output schema, the description provides a minimal but functional explanation. It lacks details about the format of the answer or limitations, which might be needed for effective use. However, it is not incomplete to the point of being unusable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters adequately. The description adds no new meaning beyond the schema, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Describes the tool as asking a specific question about a YouTube video and returning an answer. The verb 'ask' and resource 'video content' are clear. However, it does not explicitly differentiate from sibling tools like 'summarize_video', which also deals with video content, though the 'specific question' aspect provides some distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when needing to ask a question about video content, but does not provide explicit guidance on when to use this tool versus alternatives like 'summarize_video' or when not to use it. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract frames from a YouTube video at specific timestamps you provide. Use this when you already know the exact timestamps you want (e.g., from get_video_timestamps or video summary).
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| timestamps | Yes | Array of timestamps in seconds to extract frames from (e.g., [5, 30, 60, 120]) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It focuses on the extraction function but doesn't detail behavior like file output format, naming conventions, or error handling. The description adds some context beyond schema but is not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, directly stating the purpose and usage context with no fluff. It is concise and front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (4 parameters with 2 required) and no output schema, the description is adequate but not exhaustive. It covers the primary use case but lacks details on output behavior, which is acceptable since there is no output schema to complement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by clarifying the usage context (timestamps from other tools) but does not provide additional semantic meaning beyond what the schema already offers for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts frames from a YouTube video at specific timestamps. It specifies the resource (YouTube video) and the action (extract frames at timestamps), and distinguishes it from siblings like extract_screenshots by focusing on user-provided timestamps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: when you already know exact timestamps. It suggests examples like using timestamps from get_video_timestamps or video summary, implying when not to use it (if you don't have timestamps) and providing context for alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract key screenshots from a YouTube video at important moments. Uses AI to identify visually significant timestamps, then extracts frames. Returns both base64 images and optionally saves to disk.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of screenshots to extract (1-20, default: 5) | |
| output_dir | No | Optional directory to save screenshots. If not provided, uses SCREENSHOT_OUTPUT_DIR env var or temp directory. | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. | |
| resolution | No | Output resolution: thumbnail (160p), small (360p), medium (720p), large (1080p), full (original). Default: large | large |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it uses AI for timestamp selection, returns base64 images, and optionally saves to disk. With no annotations provided, this description carries the burden well. However, more details about performance or API calls could enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each providing distinct value: purpose, method, and output. Could be more concise by removing 'Optionally saves to disk' since it's covered in output_dir parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main aspects: video input, AI selection, number of screenshots, output format, and save behavior. However, no output schema means description could mention return format (array of base64 strings). Overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds context for 'focus' parameter but doesn't elaborate on 'resolution' enum meaning beyond schema. The 'output_dir' description adds context about fallback behavior, which adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it extracts screenshots from YouTube videos at important moments using AI. Distinguishes from siblings like 'extract_frames' by mentioning AI to find visually significant timestamps, but could be more specific about difference from 'extract_frames'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for extracting key screenshots, but no explicit guidance on when to use vs siblings like 'extract_frames' or 'get_video_timestamps'. No mention of prerequisites (e.g., need ffmpeg).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Preview mode: Use AI to identify important moments in a YouTube video and return their timestamps WITHOUT extracting frames. Use this to preview what timestamps would be selected before committing to extraction.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| count | No | Number of timestamps to identify (1-20, default: 5) | |
| focus | No | Optional focus for timestamp selection (e.g., 'product demos', 'code examples', 'diagrams'). Default analyzes for general key moments. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states that it uses AI to identify timestamps and does not extract frames. However, it does not disclose other behavioral traits such as API rate limits, how the AI works, or constraints on video length/format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences. The first sentence clearly defines the tool's function, and the second sentence explains its purpose relative to other tools. No unnecessary information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is fairly complete. It explains the tool's purpose and its relationship to extraction tools. However, it does not mention return format or behavior for invalid URLs, but the schema provides validation hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides descriptions for all parameters (100% coverage), so the description adds minimal value beyond the schema. However, the description does give context for the 'focus' parameter by providing examples like 'product demos' and 'code examples', which adds nuance beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to use AI to identify important moments in a YouTube video and return their timestamps without extracting frames. It distinguishes itself from sibling tools by emphasizing that it is a preview mode that does not extract frames, which sets it apart from extract_frames and extract_screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'to preview what timestamps would be selected before committing to extraction.' However, it does not explicitly mention when not to use it or suggest alternatives, but the sibling list makes it clear that extraction tools exist for different purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize a YouTube video's content. Returns a text summary based on the specified detail level.
| Name | Required | Description | Default |
|---|---|---|---|
| youtube_url | Yes | Full YouTube URL (youtube.com/watch?v=ID, youtu.be/ID, or youtube.com/shorts/ID) | |
| detail_level | No | Level of detail: brief (2-3 sentences), medium (key points with timestamps), detailed (comprehensive breakdown) | medium |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral aspects. It mentions it returns a text summary based on detail level, but does not disclose processing time, input validation, or error handling. The behavior is straightforward but minimally described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first states the core function, second explains output based on parameter. It is concise but could be more front-loaded about what the user gets. No fluff, but slightly vague on output format.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain return format. It says 'returns a text summary' but not whether it's plain text, structured, or includes timestamps. With siblings like ask_about_video, more detail on output would help. Scores 3 as it covers basic purpose but lacks output specificity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the detail_level enum values (brief, medium, detailed) with concrete expectations like 'key points with timestamps', which is not in the schema. This extra context earns a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it summarizes a YouTube video's content and returns a text summary based on a detail level. It distinguishes from siblings like ask_about_video (querying) and extract_frames (visual extraction), but could be more explicit about the summary format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives are provided. The description implies usage when a text summary is needed, but does not clarify when to choose this over ask_about_video or other tools. However, the enum options for detail_level give some usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0ask_about_videoextract_framesextract_screenshotsget_video_timestampssummarize_videoScored across 5 tools
Tools are mostly distinct: ask_about_video and summarize_video both deal with content analysis but differ in question vs. summary. extract_frames and extract_screenshots have clear differences (given timestamps vs. AI-chosen moments). get_video_timestamps is a preview for extraction, reducing overlap. Minor confusion possible between ask_about_video and summarize_video.
All tool names use verb_* pattern with snake_case (ask_about_video, extract_frames, etc.), which is consistent. Some verbs are compound (ask_about), but the pattern is predictable and clear.
5 tools is appropriate for a YouTube MCP: they cover core operations (summarize, Q&A, frame extraction, screenshot extraction, timestamp preview). Not too few or too many.
The tool set covers main user needs: summarization, Q&A, and visual extraction with a preview step. Missing operations like searching videos or managing playlists, but those may be out of scope. The workflow from preview to extraction is well-supported.
Gemini watches public YouTube videos and returns a timestamped brief or answers to your questions.
YouTube transcripts, video search, chapters and extractive summaries. No YouTube API key, no LLM.
Summarize YouTube videos or fetch transcripts. Pay-per-call via x402 (USDC on Base), no account.
Transcribe YouTube via Whisper. Summaries, chapters, semantic-search across your corpus.
