Skip to main content
Glama

video-mcp

Sort a shoot folder in one command — A-roll vs B-roll, grouped by topic, bad takes pulled into _rejects — plus a local-first video understanding MCP for Claude Desktop, Claude Code, Codex, Grok, and any MCP client. → videomcp.n3el.dev

Install (macOS)

curl -fsSL https://raw.githubusercontent.com/neelsatyavolu/videomcp/main/install.sh | bash

Or download the Mac installer, unzip it and double-click Install video-mcp.command. Either way it installs Node.js and Homebrew if missing, the video-mcp command, and ffmpeg, tesseract, whisper-cpp and a speech model (install.sh is short — read it first if you like). Sorting also needs one of the Claude Code, Codex or Grok CLIs, signed in.

The MCP server turns video into what agents already understand: metadata + timestamped transcript + key frames + timeline. The sections below cover it and other ways to install.

Related MCP server: VZT Video-Intel

Other install options

Prerequisites: Node.js 18+ and ffmpeg (brew install ffmpeg).

From a clone

git clone https://github.com/neelsatyavolu/videomcp && cd videomcp
npm install && npm run build
node dist/index.js setup      # ffmpeg + tesseract + whisper-cpp + model
node dist/index.js install    # Claude Desktop / Code / Codex / Grok

Global npm install (from the latest release)

npm install -g https://github.com/neelsatyavolu/videomcp/releases/latest/download/video-mcp-server.tgz
video-mcp setup            # ffmpeg / tesseract / whisper-cpp + model
video-mcp install          # all clients
video-mcp doctor           # check ffmpeg / Whisper / installs

Use as an MCP server

video-mcp install wires Claude Desktop, Claude Code, Codex, and Grok automatically. Restart the app / start a new agent session after installing.

video-mcp install --client desktop   # Claude Desktop only
video-mcp install --client code      # Claude Code only
video-mcp install --client codex
video-mcp install --client grok
video-mcp install --client all       # default

video-mcp status
video-mcp uninstall
video-mcp help

CLI

Command

What it does

video-mcp setup

Install needed deps: ffmpeg, tesseract, whisper-cpp + ggml model

video-mcp install

Add MCP to agent config files

video-mcp uninstall

Remove from configs

video-mcp status

Show where it's installed

video-mcp doctor

ffmpeg / whisper / tesseract + client status

video-mcp sort <folder>

Sort a shoot folder by topic, A/B roll and quality (see below)

video-mcp serve

Run MCP on stdio (also the default with no args)

Needed system tools

Tool

Role

ffmpeg

Frames, probe, audio

whisper-cpp + ggml-base.bin

Local speech-to-text

tesseract

On-screen text (OCR)

yt-dlp

YouTube/etc. (optional)

Configs are updated in place; a .bak backup is written first.

Client

Config file

Claude Desktop

~/Library/Application Support/Claude/claude_desktop_config.json

Claude Code

~/.claude.json

Codex

~/.codex/config.toml

Grok

~/.grok/config.toml

Sorting a shoot folder

video-mcp sort ~/Footage/shoot-0924 --dry-run   # see the plan, move nothing
video-mcp sort ~/Footage/shoot-0924             # asks before moving
video-mcp sort --undo ~/Footage/shoot-0924      # put everything back

Every clip is analysed on your machine (transcript, keyframes, OCR, and ffmpeg checks for focus, exposure, shake, audio clipping, silence, black and frozen frames). A signed-in agent CLI (--agent grok|codex|claude; default is the first one installed, in that order) looks at each clip and says keep or reject, then groups the whole shoot into topics and picks the best of repeated takes. Files are moved in place and never overwritten; names are kept, except that a clash gets (2) added:

shoot/
  <Topic>/a-roll/ · <Topic>/b-roll/
  _rejects/<Topic>/{a-roll,b-roll}/   bad clips, alternate takes, near-duplicates
  footage-report.md                   every clip with its verdict and reasons
  .footage-sort/                      analysis cache + undo manifest

A-roll is a clip with speech over at least 40% of its length; a folder named a-roll or b-roll overrides that. Very short, mostly black or frozen clips are rejected without asking the agent. Re-running only sorts clips added since. Options: --model, --yes, --concurrency N (default 3), --no-cache. --undo reverts the most recent run; repeat it to go further back. Speech-to-text runs locally with whisper.cpp; only if no local backend works and OPENAI_API_KEY is set is audio sent to OpenAI's Whisper API.

Tools

Tool

Use when

video_check_deps

Setup / missing ffmpeg / Whisper

video_info

Duration, resolution, codecs (fast)

video_transcribe

Speech / captions only

video_get_frame

Exact moment after transcript hit

video_get_frame_burst

Motion / UI transitions

video_extract_frames

Keyframes without full pipeline

video_analyze

“Watch / summarize / understand this video” (frames + speech + OCR)

video_ocr

On-screen text only (video keyframes or a single image)

video_search_transcript

Find topic timestamps in long videos

Prompt: watch_video — guided workflow for agents.

Optional dependencies

Tool

Why

ffmpeg

Required — frames, probe, audio

yt-dlp

YouTube / TikTok / platform URLs (brew install yt-dlp)

whisper / whisper.cpp / OPENAI_API_KEY

Transcripts when captions are missing

Manual config (if you prefer)

{
  "mcpServers": {
    "video": {
      "command": "node",
      "args": ["/ABSOLUTE/PATH/TO/videomcp/dist/index.js"]
    }
  }
}

Codex / Grok TOML:

[mcp_servers.video]
command = "node"
args = ["/ABSOLUTE/PATH/TO/videomcp/dist/index.js"]
startup_timeout_sec = 30
tool_timeout_sec = 600

Environment

Variable

Purpose

VIDEO_MCP_WORK_DIR

Cache/download dir

VIDEO_MCP_MAX_INLINE_IMAGES

Max inline thumbs (default 6)

VIDEO_MCP_WRITE_SIDECARS

1 = write <video>.videomcp.json next to local files

VIDEO_MCP_DISABLE_OCR

1 = skip tesseract

WHISPER_MODEL / WHISPER_LANGUAGE

Local Whisper

OPENAI_API_KEY

Cloud Whisper fallback

YTDLP_COOKIES

Auth for restricted URLs

TESSERACT_BIN

Override tesseract path

Dev

npm install && npm run build
npm start                 # MCP stdio
npm run smoke -- /path/to/video.mp4

License

MIT

Available Tools

9 tools
video_analyzeFull video analysisA
Read-onlyIdempotent

Most powerful one-shot tool: resolve source → probe → transcript + keyframes (parallel) → OCR on-screen text → merged timeline + agent summary.

Prefer this when the user asks to "watch", "summarize", or "understand" a video. Use detail=brief for long lectures when only speech matters; detailed for short UI demos.

Returns: at-a-glance summary, metadata, transcript, OCR text, frame paths (+ evenly spaced inline thumbs), timeline, warnings.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNo
detailNobrief=metadata+transcript only; standard=scene frames+transcript; detailed=dense framesstandard
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
end_secNo
languageNo
skip_ocrNoSkip tesseract OCR on keyframes
start_secNo
max_framesNo
skip_framesNo
force_refreshNo
inline_imagesNo
response_formatNoResponse text formatmarkdown
skip_transcriptNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds genuine behavioral context beyond that: the execution flow, parallel transcript+keyframe processing, OCR step, and merged output composition. It does not mention latency or resource cost, which is notable for a tool described as 'most powerful one-shot', but this is a minor gap given annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: pipeline overview (front-loaded first sentence), when-to-prefer routing, detail-parameter guidance, and the returns list. No fluff or repetition of schema content. For a 13-parameter tool this is tightly compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates what the tool returns. For a complex tool it covers the workflow, usage routing, and output composition well. The main completeness gap is parameter semantics for the several undocumented optional params, but those have sensible defaults and only `source` is required, so an agent can call it correctly with the information given.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 31% (4 of 13 params documented), so the description must compensate for many undocumented parameters. It adds real usage semantics for `detail` (brief for lectures, detailed for demos) and hints at `inline_images` ('evenly spaced inline thumbs'), but leaves mode, start_sec/end_sec, max_frames, skip_frames, force_refresh, language, and skip_transcript unexplained. Compensation is only partial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete multi-stage pipeline with explicit verbs ('resolve source → probe → transcript + keyframes → OCR → merged timeline'), and lists the exact return artifacts (summary, metadata, transcript, OCR text, frame paths, timeline, warnings). Calling itself the 'one-shot' comprehensive tool clearly differentiates it from siblings like video_transcribe, video_ocr, and video_extract_frames.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance ('Prefer this when the user asks to watch, summarize, or understand a video') and even maps detail levels to task types (brief for long lectures, detailed for short UI demos). However, it never names siblings as alternatives or states when NOT to use this tool, so the exclusion logic is absent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_check_depsCheck video dependenciesA
Read-onlyIdempotent

Verify ffmpeg, ffprobe, yt-dlp, and Whisper backends are available. Call first when setup fails.

Returns dependency name, availability, path, version, and install hints.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive behavior. The description adds useful context about the operation (verifying backend availability) and the shape of results (dependency name, availability, path, version, install hints). No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences; the first states the core action and the second lists the exact output fields. No filler or redundant repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless diagnostic tool with safety covered by annotations, the description provides everything an agent needs: what is checked, when to invoke it, and what the response contains. The sibling list confirms no overlapping tool requires disambiguation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the schema fully documents the interface. The description's mention of return fields is sufficient context for a parameterless tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Uses a specific verb 'Verify' and names the exact resources (ffmpeg, ffprobe, yt-dlp, Whisper backends). Clearly distinct from video-analysis, transcription, and frame-extraction siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to call ('Call first when setup fails'), which is actionable context. No exclusions or alternatives are provided, but none exist among siblings, so the guidance is adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_extract_framesExtract key framesA
Read-onlyIdempotent

Extract representative JPEG frames via scene-change detection and/or interval sampling.

Modes: scene (visual cuts), interval (uniform), both (default fallback). Returns paths always; inline images for vision clients (capped).

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNoboth
detailNobrief=metadata+transcript only; standard=scene frames+transcript; detailed=dense framesstandard
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
end_secNo
start_secNo
max_framesNo
inline_imagesNo
response_formatNoResponse text formatmarkdown

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral details: it always returns paths, optionally returns inline images for vision clients, and caps those images. It also clarifies the mode semantics (visual cuts vs uniform sampling). No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the first sentence states the core purpose, followed by a terse but informative line about modes and output behavior. Every sentence earns its place without redundancy, and the structure is easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters, only 38% schema coverage, and no output schema, the description is not complete enough. It omits explanations for time-range parameters (start_sec, end_sec), frame limits (max_frames), and the inline_images toggle. The 'capped' mention is vague without a specific limit. While annotations cover safety, the operational details are insufficient for an agent to call this tool correctly without external knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 38%, and the description does not compensate for the undocumented parameters. It explains 'mode' and touches on output behavior, but parameters like start_sec, end_sec, max_frames, and inline_images have no description in either the schema or the tool description. The agent must infer their meaning, which is a significant gap for a tool with 8 parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts representative JPEG frames via scene-change detection and/or interval sampling, which is a specific verb-resource pair. The modes (scene, interval, both) further clarify functionality, and while siblings like video_get_frame and video_get_frame_burst exist, the scene/interval distinction is unique enough to differentiate without explicit mention.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the three modes and notes 'both' as default fallback, giving some usage context. However, it does not state when to prefer this tool over siblings like video_get_frame or video_get_frame_burst, nor does it mention exclusions or prerequisites such as dependency checks (e.g., video_check_deps). The guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_get_frameGet frame at timestampA
Read-onlyIdempotent

Extract a single JPEG frame at a timestamp (seconds). Returns file path and an inline image for vision-capable clients.

Use after reading a transcript to inspect a specific moment.

ParametersJSON Schema
NameRequiredDescriptionDefault
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
time_secYesTimestamp in seconds
inline_imageNoInclude base64 image content block

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive behavior. The description adds useful behavioral context: the output is a JPEG, where it goes (file path), and that an inline image is included for vision-capable clients. This exceeds what annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no redundancy. The core action and output are first, and the usage pointer is a concise second sentence. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only tool with rich annotations, the description covers the use case, output, and key format. It doesn't explain error cases or file path details, but with no output schema and three straightforward parameters, this is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are fully documented in the schema. The description reinforces that time_sec is in seconds but doesn't need to add more; the schema already handles parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Extract a single JPEG frame at a timestamp (seconds).' The 'single' qualifier distinguishes it from siblings like video_extract_frames and video_get_frame_burst, and it also clarifies the return (file path, inline image).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit usage context: 'Use after reading a transcript to inspect a specific moment.' This tells the agent when the tool is appropriate, though it doesn't name alternatives or exclusions explicitly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_get_frame_burstBurst frames in time rangeB
Read-onlyIdempotent

Extract N frames across a short time window for motion, UI transitions, or animations.

Prefer narrow windows (e.g. 0.5–3s) with 4–16 frames.

ParametersJSON Schema
NameRequiredDescriptionDefault
countNo
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
end_secYes
start_secYes
inline_imagesNo
response_formatNoResponse text formatmarkdown

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds the window/frame-count guidance, which is heuristic rather than behavioral, and contributes little about internal behavior (e.g., fetching across openWorld sources or output handling). No contradiction with annotations exists, so a mid score is warranted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no filler: the first front-loads purpose and use cases, the second delivers tuning guidance. Every sentence earns its place; only slightly more specific parameter coverage would push it to a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and only 33% parameter coverage, the description carries heavy weight but leaves key gaps: it never explains what the response looks like in markdown vs json, the effect of inline_images, or how this tool differs from video_extract_frames. The window/frame heuristics help, but an agent still cannot predict the tool's output or its exact sibling boundary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33%, below the 50% threshold, so the description must partially compensate. It does so indirectly: '4–16 frames' maps to count and '0.5–3s' maps to the start/end window, offering useful bounds. However, it never defines inline_images or response_format behavior, and the exact semantics of start_sec/end_sec are left to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb-resource-scope ('Extract N frames across a short time window') plus three use cases (motion, UI transitions, animations). It is distinguished implicitly from the single-frame sibling video_get_frame, though it does not explicitly differentiate itself from the similarly named video_extract_frames.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides actionable heuristics ('Prefer narrow windows e.g. 0.5–3s with 4–16 frames') and implicit usage context via the listed use cases. However, it gives no explicit when-not-to-use advice and does not name or route to any alternative sibling tool, leaving selection among video_extract_frames and video_get_frame to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_infoVideo metadataA
Read-onlyIdempotent

Fast metadata probe (duration, resolution, codecs, audio presence) without transcription or frames.

Use before expensive analysis to plan detail level / time ranges.

ParametersJSON Schema
NameRequiredDescriptionDefault
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
response_formatNoResponse text formatmarkdown

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint, idempotentHint, openWorldHint, and destructiveHint false. The description adds 'fast' and the metadata-only scope, which are useful behavioral traits beyond the annotations and are not implied by them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two crisp sentences with no filler. The core purpose and scope are front-loaded, and the second sentence provides actionable usage guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only probe with two fully-documented parameters and strong annotations, the description fully equips an agent to decide when to call it and what it will return. The listed fields substitute for an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both source and response_format fully described in the schema. The description lists output metadata fields but adds no parameter semantics beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (probe) and resource (video metadata) with concrete fields (duration, resolution, codecs, audio presence). Explicitly excludes transcription and frames, distinguishing it from video_transcribe and video_extract_frames.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear when-to-use instruction: 'Use before expensive analysis to plan detail level / time ranges.' It implies exclusions via 'without transcription or frames' but does not explicitly name sibling alternatives, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_ocrOCR frames or a single imageA
Read-onlyIdempotent

Extract on-screen text with tesseract. Pass a video source to sample keyframes + OCR, or an image path for a single frame/screenshot. Requires tesseract on PATH (brew install tesseract).

ParametersJSON Schema
NameRequiredDescriptionDefault
sourceYesVideo path/URL, or absolute path to a .jpg/.png image
max_framesNo
response_formatNoResponse text formatmarkdown

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds useful operational detail beyond the annotations: it samples keyframes rather than analyzing every frame, and it calls out the external tesseract dependency with an installation hint. This helps an agent anticipate missing-dependency failures and understand that video OCR is sampled rather than exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three brief sentences cover the action, input modes, and prerequisite with no filler. The most important behavior is front-loaded, and the dependency caveat is placed at the end without disrupting the main usage guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with three parameters and rich annotations, the description covers the core behavior, input modes, and prerequisite. It does not spell out the exact JSON output structure, but the response_format parameter and 'Extract on-screen text' make the expected result clear enough for a straightforward call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds meaning for source by clarifying video keyframe sampling versus single-frame OCR. However, max_frames is left entirely to the schema's min/max/default constraints, and response_format relies on its schema description. With 67% schema coverage, the description partially compensates but still leaves some parameter behavior to inference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the specific action ('Extract on-screen text with tesseract') and clarifies the two supported input modes: video keyframe sampling and single image. This clearly distinguishes it from sibling tools like video_transcribe (speech) and video_extract_frames (frames without OCR).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete invocation guidance by telling the agent to pass either a video source or an image path. It does not explicitly name sibling alternatives or state when not to use this tool, but the intended context is clear from the two documented modes.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_search_transcriptSearch within transcriptA
Read-onlyIdempotent

Transcribe (or reuse captions) then return segments matching a query string (case-insensitive).

Useful for long videos: find when a topic is mentioned, then video_get_frame at that timestamp.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesSubstring or keywords to find
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
languageNo
response_formatNoResponse text formatmarkdown

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, covering safety. The description adds valuable behavioral nuance: it may transcribe new content or reuse existing captions, and it returns matching segments. This goes beyond the structured annotations and informs the agent about underlying processing behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero wasted words. The core behavior is front-loaded in the first sentence, and the second provides a concrete use case with a sibling tool reference. It is optimally concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the essential context: what it does, when to use it, and how to follow up (video_get_frame). It implies that returned segments include timestamps. It does not explicitly describe the exact output structure, but the response_format parameter and the reference to timestamps make it sufficient for agent use. Minor gaps like potential transcription delay are not critical.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high (75%) with query and source already well-described. The description adds the 'case-insensitive' detail for query semantics, which is not present in the schema. This enriches the parameter meaning beyond the schema alone, justifying a score above the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: transcribe (or reuse captions) and return segments matching a query string. It specifies the case-insensitive nature and distinguishes itself from sibling video_transcribe by adding search capability. The verb+resource+search operation is precise and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides specific usage context: 'Useful for long videos' and suggests pairing with video_get_frame. It implies the alternative (video_transcribe) by distinguishing the search functionality, but does not explicitly name it or state when not to use this tool. However, the guidance is clear enough for an agent to infer the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

video_transcribeTranscribe video audioA
Read-onlyIdempotent

Timestamped transcript from captions (preferred) or local/cloud Whisper.

Order: sidecar .vtt/.srt → yt-dlp captions → whisper.cpp → whisper CLI → OpenAI Whisper API. Fast for YouTube (native captions). Local files need Whisper or a sidecar caption file.

ParametersJSON Schema
NameRequiredDescriptionDefault
sourceYesAbsolute local path, file:// URI, direct media URL, or platform URL (YouTube/TikTok/etc via yt-dlp)
languageNoLanguage code hint for Whisper (e.g. en, es, pt)
force_whisperNoSkip captions and force ASR
response_formatNoResponse text formatmarkdown

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so no contradiction. The description adds behavioral context beyond these: it specifies the order of caption/Whisper sources, notes that local files require Whisper or sidecar, and states YouTube is fast due to native captions. This helps an agent set expectations about execution time and dependencies without repeating annotation information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: three short lines that front-load the core purpose and then provide the resolution order and practical tips. There is zero fluff or redundancy. Every sentence serves a purpose, from the output definition to the operational hints.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's modest complexity (4 params, all documented in schema) and the annotations covering safety (read-only, idempotent), the description is largely complete. It indicates the output is a timestamped transcript and outlines the method. It might benefit from an explicit note about the response format (markdown vs json) but that is in the schema. Overall, an agent can call this tool correctly without missing critical information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters (source, language, force_whisper, response_format). The description does not add significant per-parameter semantics beyond what the schema provides, though it does contextualize the preference order (captions vs force_whisper) and mentions Whisper for local/cloud, which aligns with the source parameter. Since the schema handles the heavy lifting, a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: producing a timestamped transcript from video audio, with a specified preference for captions or Whisper. It uses a specific verb ('transcribe') and resource ('video audio'), and the output type is explicitly named. It naturally differentiates from sibling tools like video_analyze or video_ocr, which are not transcription-focused.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides practical guidance on when the tool is efficient (fast for YouTube) and requirements for local files (need Whisper or sidecar captions). It also outlines the source resolution order. However, it does not explicitly mention when to use an alternative tool (e.g., if visual analysis is needed), but the purpose is clear enough that an agent can infer suitability. This is strong guidance, missing only direct exclusion of alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 9 tool updatesv1.3.0
    • First observedvideo_analyze
    • First observedvideo_check_deps
    • First observedvideo_extract_frames
    • First observedvideo_get_frame
    • First observedvideo_get_frame_burst
    • First observedvideo_info
    • First observedvideo_ocr
    • First observedvideo_search_transcript
    • First observedvideo_transcribe

TDQS

A4/5.0

Scored across 9 tools

Disambiguation4/5

Each tool has a clear primary role: analyze for full pipeline, transcribe for speech, ocr for text, frames for visuals, info for metadata, search for locating moments, and deps for setup. The only overlap is video_analyze, which combines several primitives, but its description explicitly directs when to use it, so agents should not be misled.

Naming Consistency4/5

All tools share the video_ prefix and use lowercase, underscore-separated action-oriented names such as video_extract_frames and video_get_frame_burst. The main deviation is video_info, which uses a noun rather than a verb, but the pattern is otherwise consistent.

Tool Count5/5

Nine tools is well-scoped for a video-processing MCP server. Each tool covers a distinct part of the video-understanding workflow, from dependency checking and metadata probing to transcription, OCR, frame extraction, and combined analysis, without unnecessary bloat.

Completeness5/5

The surface covers all major video analysis needs: metadata, transcripts, searching, frame extraction, OCR, and a one-shot summary. There are no obvious dead ends—dependencies are checked, and tools are designed to feed into each other (e.g., transcript timestamps to video_get_frame).

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • -
    license
    B
    quality
    Not graded
    maintenance
    Enables comprehensive video file metadata management including reading, editing, and batch processing of video properties like title, description, tags, and technical specifications. Supports multiple video formats with intelligent caching and search capabilities.
    8
    9 npm
    1
    -
  • A
    license
    Not graded
    quality
    C
    maintenance
    Self-hosted video intelligence pipeline that turns video into structured, citable, queryable JSON scene graphs, enabling AI assistants to analyze and search across video libraries without re-processing.
    5 npm
    1
    MIT