video-mcp
Provides cloud speech-to-text transcription through OpenAI's Whisper API as a fallback when local Whisper transcription is unavailable.
Allows the server to fetch and analyze TikTok videos via yt-dlp, including transcripts, keyframes, OCR, and technical quality checks.
Allows the server to fetch and analyze YouTube videos via yt-dlp, including transcripts, keyframes, OCR, and technical quality checks.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@video-mcpsort my shoot folder ~/Footage/shoot-0924 by topic and quality"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
video-mcp
Sort a shoot folder in one command — A-roll vs B-roll, grouped by topic, bad takes pulled into
_rejects — plus a local-first video understanding MCP for Claude Desktop, Claude Code, Codex,
Grok, and any MCP client. → videomcp.n3el.dev
Install (macOS)
curl -fsSL https://raw.githubusercontent.com/neelsatyavolu/videomcp/main/install.sh | bashOr download the Mac installer,
unzip it and double-click Install video-mcp.command. Either way it installs Node.js and Homebrew
if missing, the video-mcp command, and ffmpeg, tesseract, whisper-cpp and a speech model
(install.sh is short — read it first if you like). Sorting also needs one of the
Claude Code, Codex or Grok CLIs, signed in.
The MCP server turns video into what agents already understand: metadata + timestamped transcript + key frames + timeline. The sections below cover it and other ways to install.
Related MCP server: VZT Video-Intel
Other install options
Prerequisites: Node.js 18+ and ffmpeg (brew install ffmpeg).
From a clone
git clone https://github.com/neelsatyavolu/videomcp && cd videomcp
npm install && npm run build
node dist/index.js setup # ffmpeg + tesseract + whisper-cpp + model
node dist/index.js install # Claude Desktop / Code / Codex / GrokGlobal npm install (from the latest release)
npm install -g https://github.com/neelsatyavolu/videomcp/releases/latest/download/video-mcp-server.tgz
video-mcp setup # ffmpeg / tesseract / whisper-cpp + model
video-mcp install # all clients
video-mcp doctor # check ffmpeg / Whisper / installsUse as an MCP server
video-mcp install wires Claude Desktop, Claude Code, Codex, and Grok automatically. Restart the app / start a new agent session after installing.
video-mcp install --client desktop # Claude Desktop only
video-mcp install --client code # Claude Code only
video-mcp install --client codex
video-mcp install --client grok
video-mcp install --client all # default
video-mcp status
video-mcp uninstall
video-mcp helpCLI
Command | What it does |
| Install needed deps: ffmpeg, tesseract, whisper-cpp + ggml model |
| Add MCP to agent config files |
| Remove from configs |
| Show where it's installed |
| ffmpeg / whisper / tesseract + client status |
| Sort a shoot folder by topic, A/B roll and quality (see below) |
| Run MCP on stdio (also the default with no args) |
Needed system tools
Tool | Role |
ffmpeg | Frames, probe, audio |
whisper-cpp + ggml-base.bin | Local speech-to-text |
tesseract | On-screen text (OCR) |
yt-dlp | YouTube/etc. (optional) |
Configs are updated in place; a .bak backup is written first.
Client | Config file |
Claude Desktop |
|
Claude Code |
|
Codex |
|
Grok |
|
Sorting a shoot folder
video-mcp sort ~/Footage/shoot-0924 --dry-run # see the plan, move nothing
video-mcp sort ~/Footage/shoot-0924 # asks before moving
video-mcp sort --undo ~/Footage/shoot-0924 # put everything backEvery clip is analysed on your machine (transcript, keyframes, OCR, and ffmpeg checks for focus,
exposure, shake, audio clipping, silence, black and frozen frames). A signed-in agent CLI
(--agent grok|codex|claude; default is the first one installed, in that order) looks at each
clip and says keep or reject, then groups the whole shoot into topics and picks the best of
repeated takes. Files are moved in place and never overwritten; names are kept, except that a
clash gets (2) added:
shoot/
<Topic>/a-roll/ · <Topic>/b-roll/
_rejects/<Topic>/{a-roll,b-roll}/ bad clips, alternate takes, near-duplicates
footage-report.md every clip with its verdict and reasons
.footage-sort/ analysis cache + undo manifestA-roll is a clip with speech over at least 40% of its length; a folder named a-roll or
b-roll overrides that. Very short, mostly black or frozen clips are rejected without asking
the agent. Re-running only sorts clips added since. Options: --model, --yes,
--concurrency N (default 3), --no-cache. --undo reverts the most recent run; repeat it to
go further back. Speech-to-text runs locally with whisper.cpp; only if no local backend works and
OPENAI_API_KEY is set is audio sent to OpenAI's Whisper API.
Tools
Tool | Use when |
| Setup / missing ffmpeg / Whisper |
| Duration, resolution, codecs (fast) |
| Speech / captions only |
| Exact moment after transcript hit |
| Motion / UI transitions |
| Keyframes without full pipeline |
| “Watch / summarize / understand this video” (frames + speech + OCR) |
| On-screen text only (video keyframes or a single image) |
| Find topic timestamps in long videos |
Prompt: watch_video — guided workflow for agents.
Optional dependencies
Tool | Why |
ffmpeg | Required — frames, probe, audio |
yt-dlp | YouTube / TikTok / platform URLs ( |
whisper / whisper.cpp / | Transcripts when captions are missing |
Manual config (if you prefer)
{
"mcpServers": {
"video": {
"command": "node",
"args": ["/ABSOLUTE/PATH/TO/videomcp/dist/index.js"]
}
}
}Codex / Grok TOML:
[mcp_servers.video]
command = "node"
args = ["/ABSOLUTE/PATH/TO/videomcp/dist/index.js"]
startup_timeout_sec = 30
tool_timeout_sec = 600Environment
Variable | Purpose |
| Cache/download dir |
| Max inline thumbs (default 6) |
|
|
|
|
| Local Whisper |
| Cloud Whisper fallback |
| Auth for restricted URLs |
| Override tesseract path |
Dev
npm install && npm run build
npm start # MCP stdio
npm run smoke -- /path/to/video.mp4License
MIT
Available Tools
9 toolsvideo_analyzeFull video analysisARead-onlyIdempotent
Most powerful one-shot tool: resolve source → probe → transcript + keyframes (parallel) → OCR on-screen text → merged timeline + agent summary.
Prefer this when the user asks to "watch", "summarize", or "understand" a video. Use detail=brief for long lectures when only speech matters; detailed for short UI demos.
Returns: at-a-glance summary, metadata, transcript, OCR text, frame paths (+ evenly spaced inline thumbs), timeline, warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | ||
| detail | No | brief=metadata+transcript only; standard=scene frames+transcript; detailed=dense frames | standard |
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| end_sec | No | ||
| language | No | ||
| skip_ocr | No | Skip tesseract OCR on keyframes | |
| start_sec | No | ||
| max_frames | No | ||
| skip_frames | No | ||
| force_refresh | No | ||
| inline_images | No | ||
| response_format | No | Response text format | markdown |
| skip_transcript | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds genuine behavioral context beyond that: the execution flow, parallel transcript+keyframe processing, OCR step, and merged output composition. It does not mention latency or resource cost, which is notable for a tool described as 'most powerful one-shot', but this is a minor gap given annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each earning its place: pipeline overview (front-loaded first sentence), when-to-prefer routing, detail-parameter guidance, and the returns list. No fluff or repetition of schema content. For a 13-parameter tool this is tightly compressed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully enumerates what the tool returns. For a complex tool it covers the workflow, usage routing, and output composition well. The main completeness gap is parameter semantics for the several undocumented optional params, but those have sensible defaults and only `source` is required, so an agent can call it correctly with the information given.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 31% (4 of 13 params documented), so the description must compensate for many undocumented parameters. It adds real usage semantics for `detail` (brief for lectures, detailed for demos) and hints at `inline_images` ('evenly spaced inline thumbs'), but leaves mode, start_sec/end_sec, max_frames, skip_frames, force_refresh, language, and skip_transcript unexplained. Compensation is only partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a concrete multi-stage pipeline with explicit verbs ('resolve source → probe → transcript + keyframes → OCR → merged timeline'), and lists the exact return artifacts (summary, metadata, transcript, OCR text, frame paths, timeline, warnings). Calling itself the 'one-shot' comprehensive tool clearly differentiates it from siblings like video_transcribe, video_ocr, and video_extract_frames.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance ('Prefer this when the user asks to watch, summarize, or understand a video') and even maps detail levels to task types (brief for long lectures, detailed for short UI demos). However, it never names siblings as alternatives or states when NOT to use this tool, so the exclusion logic is absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_check_depsCheck video dependenciesARead-onlyIdempotent
Verify ffmpeg, ffprobe, yt-dlp, and Whisper backends are available. Call first when setup fails.
Returns dependency name, availability, path, version, and install hints.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only, idempotent, non-destructive behavior. The description adds useful context about the operation (verifying backend availability) and the shape of results (dependency name, availability, path, version, install hints). No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences; the first states the core action and the second lists the exact output fields. No filler or redundant repetition of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless diagnostic tool with safety covered by annotations, the description provides everything an agent needs: what is checked, when to invoke it, and what the response contains. The sibling list confirms no overlapping tool requires disambiguation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the schema fully documents the interface. The description's mention of return fields is sufficient context for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Uses a specific verb 'Verify' and names the exact resources (ffmpeg, ffprobe, yt-dlp, Whisper backends). Clearly distinct from video-analysis, transcription, and frame-extraction siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call ('Call first when setup fails'), which is actionable context. No exclusions or alternatives are provided, but none exist among siblings, so the guidance is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_extract_framesExtract key framesARead-onlyIdempotent
Extract representative JPEG frames via scene-change detection and/or interval sampling.
Modes: scene (visual cuts), interval (uniform), both (default fallback). Returns paths always; inline images for vision clients (capped).
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | both | |
| detail | No | brief=metadata+transcript only; standard=scene frames+transcript; detailed=dense frames | standard |
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| end_sec | No | ||
| start_sec | No | ||
| max_frames | No | ||
| inline_images | No | ||
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral details: it always returns paths, optionally returns inline images for vision clients, and caps those images. It also clarifies the mode semantics (visual cuts vs uniform sampling). No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the first sentence states the core purpose, followed by a terse but informative line about modes and output behavior. Every sentence earns its place without redundancy, and the structure is easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters, only 38% schema coverage, and no output schema, the description is not complete enough. It omits explanations for time-range parameters (start_sec, end_sec), frame limits (max_frames), and the inline_images toggle. The 'capped' mention is vague without a specific limit. While annotations cover safety, the operational details are insufficient for an agent to call this tool correctly without external knowledge.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 38%, and the description does not compensate for the undocumented parameters. It explains 'mode' and touches on output behavior, but parameters like start_sec, end_sec, max_frames, and inline_images have no description in either the schema or the tool description. The agent must infer their meaning, which is a significant gap for a tool with 8 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts representative JPEG frames via scene-change detection and/or interval sampling, which is a specific verb-resource pair. The modes (scene, interval, both) further clarify functionality, and while siblings like video_get_frame and video_get_frame_burst exist, the scene/interval distinction is unique enough to differentiate without explicit mention.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the three modes and notes 'both' as default fallback, giving some usage context. However, it does not state when to prefer this tool over siblings like video_get_frame or video_get_frame_burst, nor does it mention exclusions or prerequisites such as dependency checks (e.g., video_check_deps). The guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_get_frameGet frame at timestampARead-onlyIdempotent
Extract a single JPEG frame at a timestamp (seconds). Returns file path and an inline image for vision-capable clients.
Use after reading a transcript to inspect a specific moment.
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| time_sec | Yes | Timestamp in seconds | |
| inline_image | No | Include base64 image content block |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive behavior. The description adds useful behavioral context: the output is a JPEG, where it goes (file path), and that an inline image is included for vision-capable clients. This exceeds what annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no redundancy. The core action and output are first, and the usage pointer is a concise second sentence. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with rich annotations, the description covers the use case, output, and key format. It doesn't explain error cases or file path details, but with no output schema and three straightforward parameters, this is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description reinforces that time_sec is in seconds but doesn't need to add more; the schema already handles parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Extract a single JPEG frame at a timestamp (seconds).' The 'single' qualifier distinguishes it from siblings like video_extract_frames and video_get_frame_burst, and it also clarifies the return (file path, inline image).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage context: 'Use after reading a transcript to inspect a specific moment.' This tells the agent when the tool is appropriate, though it doesn't name alternatives or exclusions explicitly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_get_frame_burstBurst frames in time rangeBRead-onlyIdempotent
Extract N frames across a short time window for motion, UI transitions, or animations.
Prefer narrow windows (e.g. 0.5–3s) with 4–16 frames.
| Name | Required | Description | Default |
|---|---|---|---|
| count | No | ||
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| end_sec | Yes | ||
| start_sec | Yes | ||
| inline_images | No | ||
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds the window/frame-count guidance, which is heuristic rather than behavioral, and contributes little about internal behavior (e.g., fetching across openWorld sources or output handling). No contradiction with annotations exists, so a mid score is warranted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no filler: the first front-loads purpose and use cases, the second delivers tuning guidance. Every sentence earns its place; only slightly more specific parameter coverage would push it to a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and only 33% parameter coverage, the description carries heavy weight but leaves key gaps: it never explains what the response looks like in markdown vs json, the effect of inline_images, or how this tool differs from video_extract_frames. The window/frame heuristics help, but an agent still cannot predict the tool's output or its exact sibling boundary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%, below the 50% threshold, so the description must partially compensate. It does so indirectly: '4–16 frames' maps to count and '0.5–3s' maps to the start/end window, offering useful bounds. However, it never defines inline_images or response_format behavior, and the exact semantics of start_sec/end_sec are left to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb-resource-scope ('Extract N frames across a short time window') plus three use cases (motion, UI transitions, animations). It is distinguished implicitly from the single-frame sibling video_get_frame, though it does not explicitly differentiate itself from the similarly named video_extract_frames.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides actionable heuristics ('Prefer narrow windows e.g. 0.5–3s with 4–16 frames') and implicit usage context via the listed use cases. However, it gives no explicit when-not-to-use advice and does not name or route to any alternative sibling tool, leaving selection among video_extract_frames and video_get_frame to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_infoVideo metadataARead-onlyIdempotent
Fast metadata probe (duration, resolution, codecs, audio presence) without transcription or frames.
Use before expensive analysis to plan detail level / time ranges.
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnlyHint, idempotentHint, openWorldHint, and destructiveHint false. The description adds 'fast' and the metadata-only scope, which are useful behavioral traits beyond the annotations and are not implied by them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two crisp sentences with no filler. The core purpose and scope are front-loaded, and the second sentence provides actionable usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only probe with two fully-documented parameters and strong annotations, the description fully equips an agent to decide when to call it and what it will return. The listed fields substitute for an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both source and response_format fully described in the schema. The description lists output metadata fields but adds no parameter semantics beyond the schema, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (probe) and resource (video metadata) with concrete fields (duration, resolution, codecs, audio presence). Explicitly excludes transcription and frames, distinguishing it from video_transcribe and video_extract_frames.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear when-to-use instruction: 'Use before expensive analysis to plan detail level / time ranges.' It implies exclusions via 'without transcription or frames' but does not explicitly name sibling alternatives, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_ocrOCR frames or a single imageARead-onlyIdempotent
Extract on-screen text with tesseract. Pass a video source to sample keyframes + OCR, or an image path for a single frame/screenshot. Requires tesseract on PATH (brew install tesseract).
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Video path/URL, or absolute path to a .jpg/.png image | |
| max_frames | No | ||
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds useful operational detail beyond the annotations: it samples keyframes rather than analyzing every frame, and it calls out the external tesseract dependency with an installation hint. This helps an agent anticipate missing-dependency failures and understand that video OCR is sampled rather than exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three brief sentences cover the action, input modes, and prerequisite with no filler. The most important behavior is front-loaded, and the dependency caveat is placed at the end without disrupting the main usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with three parameters and rich annotations, the description covers the core behavior, input modes, and prerequisite. It does not spell out the exact JSON output structure, but the response_format parameter and 'Extract on-screen text' make the expected result clear enough for a straightforward call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for source by clarifying video keyframe sampling versus single-frame OCR. However, max_frames is left entirely to the schema's min/max/default constraints, and response_format relies on its schema description. With 67% schema coverage, the description partially compensates but still leaves some parameter behavior to inference.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the specific action ('Extract on-screen text with tesseract') and clarifies the two supported input modes: video keyframe sampling and single image. This clearly distinguishes it from sibling tools like video_transcribe (speech) and video_extract_frames (frames without OCR).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives concrete invocation guidance by telling the agent to pass either a video source or an image path. It does not explicitly name sibling alternatives or state when not to use this tool, but the intended context is clear from the two documented modes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_search_transcriptSearch within transcriptARead-onlyIdempotent
Transcribe (or reuse captions) then return segments matching a query string (case-insensitive).
Useful for long videos: find when a topic is mentioned, then video_get_frame at that timestamp.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Substring or keywords to find | |
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| language | No | ||
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, covering safety. The description adds valuable behavioral nuance: it may transcribe new content or reuse existing captions, and it returns matching segments. This goes beyond the structured annotations and informs the agent about underlying processing behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero wasted words. The core behavior is front-loaded in the first sentence, and the second provides a concrete use case with a sibling tool reference. It is optimally concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential context: what it does, when to use it, and how to follow up (video_get_frame). It implies that returned segments include timestamps. It does not explicitly describe the exact output structure, but the response_format parameter and the reference to timestamps make it sufficient for agent use. Minor gaps like potential transcription delay are not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high (75%) with query and source already well-described. The description adds the 'case-insensitive' detail for query semantics, which is not present in the schema. This enriches the parameter meaning beyond the schema alone, justifying a score above the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: transcribe (or reuse captions) and return segments matching a query string. It specifies the case-insensitive nature and distinguishes itself from sibling video_transcribe by adding search capability. The verb+resource+search operation is precise and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides specific usage context: 'Useful for long videos' and suggests pairing with video_get_frame. It implies the alternative (video_transcribe) by distinguishing the search functionality, but does not explicitly name it or state when not to use this tool. However, the guidance is clear enough for an agent to infer the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
video_transcribeTranscribe video audioARead-onlyIdempotent
Timestamped transcript from captions (preferred) or local/cloud Whisper.
Order: sidecar .vtt/.srt → yt-dlp captions → whisper.cpp → whisper CLI → OpenAI Whisper API. Fast for YouTube (native captions). Local files need Whisper or a sidecar caption file.
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Absolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp) | |
| language | No | Language code hint for Whisper (e.g. en, es, pt) | |
| force_whisper | No | Skip captions and force ASR | |
| response_format | No | Response text format | markdown |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so no contradiction. The description adds behavioral context beyond these: it specifies the order of caption/Whisper sources, notes that local files require Whisper or sidecar, and states YouTube is fast due to native captions. This helps an agent set expectations about execution time and dependencies without repeating annotation information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: three short lines that front-load the core purpose and then provide the resolution order and practical tips. There is zero fluff or redundancy. Every sentence serves a purpose, from the output definition to the operational hints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's modest complexity (4 params, all documented in schema) and the annotations covering safety (read-only, idempotent), the description is largely complete. It indicates the output is a timestamped transcript and outlines the method. It might benefit from an explicit note about the response format (markdown vs json) but that is in the schema. Overall, an agent can call this tool correctly without missing critical information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (source, language, force_whisper, response_format). The description does not add significant per-parameter semantics beyond what the schema provides, though it does contextualize the preference order (captions vs force_whisper) and mentions Whisper for local/cloud, which aligns with the source parameter. Since the schema handles the heavy lifting, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: producing a timestamped transcript from video audio, with a specified preference for captions or Whisper. It uses a specific verb ('transcribe') and resource ('video audio'), and the output type is explicitly named. It naturally differentiates from sibling tools like video_analyze or video_ocr, which are not transcription-focused.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides practical guidance on when the tool is efficient (fast for YouTube) and requirements for local files (need Whisper or sidecar captions). It also outlines the source resolution order. However, it does not explicitly mention when to use an alternative tool (e.g., if visual analysis is needed), but the purpose is clear enough that an agent can infer suitability. This is strong guidance, missing only direct exclusion of alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
9 tool updates
v1.3.0- First observed
video_analyze - First observed
video_check_deps - First observed
video_extract_frames - First observed
video_get_frame - First observed
video_get_frame_burst - First observed
video_info - First observed
video_ocr - First observed
video_search_transcript - First observed
video_transcribe
TDQS
Scored across 9 tools
Each tool has a clear primary role: analyze for full pipeline, transcribe for speech, ocr for text, frames for visuals, info for metadata, search for locating moments, and deps for setup. The only overlap is video_analyze, which combines several primitives, but its description explicitly directs when to use it, so agents should not be misled.
All tools share the video_ prefix and use lowercase, underscore-separated action-oriented names such as video_extract_frames and video_get_frame_burst. The main deviation is video_info, which uses a noun rather than a verb, but the pattern is otherwise consistent.
Nine tools is well-scoped for a video-processing MCP server. Each tool covers a distinct part of the video-understanding workflow, from dependency checking and metadata probing to transcription, OCR, frame extraction, and combined analysis, without unnecessary bloat.
The surface covers all major video analysis needs: metadata, transcripts, searching, frame extraction, OCR, and a one-shot summary. There are no obvious dead ends—dependencies are checked, and tools are designed to feed into each other (e.g., transcript timestamps to video_get_frame).
Maintenance
Related MCP Connectors
Edit video by talking to your AI — search footage, cut timelines, apply effects, add captions.
Understand your videos with Reka AI — search, ask questions, and extract insights.
Video analysis AI: transcripts, summaries, visual scenes/shots, clips, answers in natural language.
Personal asset search engine: everything you make or upload is searchable by what's inside it.
Related MCP Servers
- -licenseBqualityNot gradedmaintenanceEnables comprehensive video file metadata management including reading, editing, and batch processing of video properties like title, description, tags, and technical specifications. Supports multiple video formats with intelligent caching and search capabilities.89 npm1-

VZT Video-Intelofficial
AlicenseNot gradedqualityCmaintenanceSelf-hosted video intelligence pipeline that turns video into structured, citable, queryable JSON scene graphs, enabling AI assistants to analyze and search across video libraries without re-processing.5 npm1MIT- AlicenseBqualityCmaintenanceEnables AI agents to edit videos locally by inspecting media, applying templates, cutting clips, and delivering finished files, with memory of human preferences.183Apache 2.0
- AlicenseNot gradedqualityBmaintenanceEnables LLM agents to process local videos into timestamped, citable text documents and then query them through tools for listing videos, retrieving transcripts, and fetching specific segments, all fully offline.MIT