mcp-video-analyzer
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
Video analysis MCP server. Extracts transcripts, key frames, metadata, comments, OCR text, and annotated timelines from video URLs and local video files.
AUTOMATIC BEHAVIOR — Do NOT wait for the user to ask:
When the user shares a video URL or local video file path → call analyze_video immediately (detail="standard").
When discussing a specific moment in a video → call analyze_moment with the time range.
When the user asks "what happens at X:XX" → call get_frame_at or analyze_moment.
When the user asks to compare frames or see motion → call get_frame_burst.
When the transcript alone would answer the question → call get_transcript (faster, no download).
When only metadata is needed (title, duration, comments) → call get_metadata (no download).
The AI should ALWAYS call the appropriate tool automatically — never ask "would you like me to analyze this video?" Just do it.
Supported sources:
Loom (loom.com/share/...) — transcript, metadata, comments, frames (no auth needed)
YouTube (watch/shorts/live/youtu.be), Vimeo, TikTok, Instagram, X/Twitter, Twitch, Dailymotion, Facebook — requires yt-dlp installed. Native captions preferred (uploaded > auto-generated), Whisper fallback otherwise; metadata includes uploader/views/chapters; no comments. Instagram and age-restricted videos usually need cookies (YTDLP_COOKIES_FROM_BROWSER=chrome or YTDLP_COOKIES=).
Direct video URLs (.mp4, .webm, .mov) — frame extraction, duration probing. When TWELVELABS_API_KEY is set, TwelveLabs Pegasus also provides an AI-generated, timestamped transcript (best-effort, not deterministic ASR) + AI summary for these (which direct URLs otherwise lack); prefer get_transcript for a text-only, no-frames answer.
Local video files — pass an absolute path (e.g., "/Users/you/clip.mp4") or a file:// URI; frame extraction + Whisper transcription work the same way
A silent-but-present audio track (common in muted Reels/Stories) is detected before transcription: the transcript comes back empty with a warning saying so — that is content, not an error.
Tools (choose the most efficient one for the task):
analyze_video: Full analysis. Use by default when a video URL appears. Returns transcript + frames + metadata + OCR + timeline.
detail="brief" → fast, metadata + truncated transcript, no video download
detail="standard" → default, scene-change frames + full transcript + OCR
detail="detailed" → dense 1fps sampling, more frames, thorough OCR
fields=["metadata","transcript"] → return only specific fields
Cached for 10min — use forceRefresh=true to re-analyze.
analyze_videos: Batch version of analyze_video. Use when given a list of sources (e.g. a folder of local files). Runs them with a concurrency limit and returns one structured result per source (counts + warnings, or a per-item error). Pair with MCP_WRITE_SIDECARS=1 for resumable bulk processing.
get_transcript: Transcript only. Faster than analyze_video when you only need what was said. Whisper fallback for videos without native transcripts.
get_metadata: Metadata + comments + chapters. No video download needed.
get_frames: Frames only (scene-change or dense=true for 1fps). Use when you need visuals without transcript.
get_frame_at: Single frame at a timestamp. Use when the transcript reveals an interesting moment and you want to see it.
get_frame_burst: N frames in a narrow time range. Use for motion, animations, fast UI changes.
analyze_moment: Deep-dive on a time range. Combines burst frames + filtered transcript + OCR + mini-timeline. Use when the user asks about a specific part of the video.
Decision flow:
User shares a video URL → analyze_video (standard)
User asks about a specific timestamp → analyze_moment or get_frame_at
User asks "what did they say about X" → get_transcript (fast, no download)
User asks "how long is this video" → get_metadata (fast, no download)
User asks for more detail after initial analysis → analyze_video (detailed) or analyze_moment
User asks to see motion/animation → get_frame_burst
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| logging | {} |
| completions | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| analyze_videoA | Analyze a video URL to extract transcript, key frames, metadata, comments, OCR text, and annotated timeline. Returns structured data about the video content:
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). Detail levels:
Use options.fields to request only specific data (e.g., ["metadata", "transcript"]). Use options.forceRefresh to bypass the cache. Use options.model / options.language / options.initialPrompt to override Whisper transcription per call (e.g. a heavier model + a domain glossary for hard audio) without restarting the server. |
| analyze_videosA | Batch-analyze many videos in one call, with a concurrency limit and per-item results. For each source it runs the same pipeline as analyze_video (frames + OCR + transcript + timeline), reusing the shared cache and on-disk sidecars. Designed for processing a corpus of local files: pair it with MCP_WRITE_SIDECARS=1 so results persist next to each video and a re-run resumes instead of recomputing. Returns a JSON summary plus one structured entry per source:
To keep the response bounded, frame images are NOT inlined and full transcript/OCR/timeline arrays are returned only when options.fields is set; otherwise you get counts. Use analyze_video on an individual source when you need the images or full data inline. |
| get_frame_atA | Extract a single video frame at a specific timestamp. Useful for inspecting what's on screen at a particular moment. The AI reads the transcript, identifies a critical moment, and requests the exact frame at that timestamp. Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). Args:
Returns: A single image of the video frame at the specified timestamp. |
| get_frame_burstA | Extract up to the requested number of source frames across a time range. Designed for motion and vibration analysis where scene-change detection fails because the "scene" doesn't change — only the position/state of objects does. Example: get_frame_burst(url, "0:15", "0:17", 10) → 10 frames in 2 seconds
Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). Args:
Returns: A JSON text block with { frameCount, from, to, warnings, frames: [{ time, timingOrigin }] }, followed by images in the same order as frames. timingOrigin is "source-pts" for measured video-relative FFmpeg timestamps and "seek-target" for requested browser seeks. Empty results include frames: []. Up to N source-frame images sampled in [from, to) by ffmpeg, without duplicating frames to meet the requested count. Fractional bounds such as "0:01.1" are supported. Browser fallback seeks across the range, including its endpoints; those seek targets are not measured source-frame PTS. |
| get_transcriptA | Extract only the transcript from a video URL. Returns timestamped transcript entries with speaker identification (when available). Faster than analyze_video when you only need the transcript. If the platform has no native transcript, attempts Whisper fallback transcription (requires @huggingface/transformers, whisper CLI, or OPENAI_API_KEY). Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp; native captions preferred), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). For local files a sidecar .vtt/.srt next to the file is used first, then an embedded subtitle track, and only then the Whisper fallback if neither exists. |
| get_metadataA | Get video metadata, comments, chapters, and AI summary from a video URL. Returns structured metadata without downloading the video or extracting frames. Faster than analyze_video when you only need metadata. Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). |
| get_framesA | Extract key frames from a video URL without transcript or metadata. Two extraction modes:
Returns optimized, deduplicated JPEG frames. Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). |
| analyze_momentA | Deep-dive analysis of a specific time range in a video. Combines burst frame extraction + transcript filtering + OCR + annotated timeline for a focused segment of the video. Use this when you need to understand exactly what happens between two timestamps:
Example: analyze_moment(url, "1:30", "2:00", 10) → 10 frames + transcript + OCR for that 30s window Supports: Loom (loom.com/share/...), YouTube/Vimeo/TikTok/Instagram/X/Twitch/Dailymotion/Facebook (requires yt-dlp), direct video URLs (.mp4, .webm, .mov), and local video files (absolute path or file:// URI). |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 8 tools
Each tool has a mostly distinct purpose: analyze_video (full pipeline), get_metadata, get_transcript, get_frames, get_frame_at, get_frame_burst, analyze_moment, and analyze_videos. The main risk is overlap between analyze_moment and a manual combination of get_frame_burst + get_transcript, and between get_frames and get_frame_burst, but the descriptions clearly differentiate sampling modes and use cases.
All names use consistent snake_case with two coherent verb families: analyze_* for heavy processing and get_* for targeted extraction. The only slight wrinkle is analyze_video vs analyze_videos (singular/plural batch pair), which is a conventional but potentially confusing distinction.
8 tools is well-scoped for a video analysis server. Each tool earns its place by covering a distinct granularity (whole video, moment, single frame, frame burst, key frames, metadata, transcript, batch).
The surface covers the domain thoroughly: full analysis, metadata, transcript, multiple frame extraction modes, focused segment analysis, and batch processing. Minor gaps like a standalone OCR-only or audio-only extraction exist, but the main video-analysis workflows have no dead ends.