Skip to main content
Glama

video-vision-mcp

CI PyPI Python License: MIT

An MCP server that gives Claude Code the ability to analyze any video — a local file or a URL — through one set of tools.

Claude can't watch video natively (only text + the first frame of an image). This server converts a video into sampled frame images + an audio transcript, or — when a Gemini key is present — a native Gemini analysis of the whole video.

It is standalone: give it a ready video (a local path or a direct URL) and it does the rest. It does not connect to Jira/Slack/etc. If a video lives behind an integration, fetch it with that integration first (download to a file or get a direct URL), then hand the file_path or url to this server.

Scenario: a Jira bug ticket has only a screen-recording, no text. Your Jira MCP downloads the attachment to a temp file → analyze_video file_path=/tmp/bug.mp4 → you see the frames + transcript (or Gemini's analysis) and can reason about the bug.

Three backend tiers (auto-selected)

Tier

Needs

What it does

1 — local (default)

nothing

ffmpeg frames + whisper.cpp transcript. Free, fully local, always works.

2 — cloud ASR

OPENAI_API_KEY or GROQ_API_KEY

Local frames, but transcription via OpenAI Whisper / Groq for higher quality.

3 — native Gemini

GEMINI_API_KEY

Gemini ingests the whole video (visual + audio) in one call, with MM:SS timestamps. Default when the key is set.

Precedence: Gemini > OpenAI > Groq > local. Set VIDEO_MCP_DISABLE_GEMINI=true to force tiers 1/2 even with a Gemini key. The backend used is named in every result.

Privacy: tier 1 never uploads anything. Tiers 2/3 print a one-time notice in the session the first time video content is sent to a third party.

Related MCP server: mcp-video

Tools

  • analyze_video — frames + transcript + metadata (the main tool). frame_interval sets seconds between frames (default 1.0; e.g. 0.5/0.25/0.1 denser, 2/5 sparser).

  • get_video_transcript_only — transcript text only.

  • extract_frames_at — frames at specific timestamps ("00:42", "1:05", 12.5).

  • list_recent_analyses — cached analyses + backend used.

Install

Requires Python ≥ 3.10. A single install pulls everything — backends, plus the ffmpeg and whisper.cpp dependencies. Nothing is ever installed globally on your machine (no brew/apt/winget, no sudo).

With uv you don't install it explicitly — uvx runs the published package on demand (see Register in Claude Code). To install into an environment instead:

uv pip install video-vision-mcp     # or: pip install video-vision-mcp

From source (development)

git clone https://github.com/KitDevUA/video-vision-mcp.git
cd video-vision-mcp
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"          # all backends bundled

Dependencies — fully self-contained

  • ffmpeg / ffprobe: if they are already on your PATH, those system binaries are used. Otherwise the bundled static-ffmpeg package supplies them (fetched once into its own local cache — never a system-wide install).

  • whisper.cpp (tier 1 transcription): shipped as the bundled pywhispercpp binding (prebuilt wheels; builds from source only if no wheel exists for your platform/Python). A whisper-cli already on PATH is used if present.

  • whisper model: the ggml model (base by default) downloads from Hugging Face into the cache on first transcription. Override with VIDEO_MCP_WHISPER_MODEL (tiny/base/small/medium/large-v3) or VIDEO_MCP_WHISPER_MODEL_PATH.

  • cloud-only: set OPENAI_API_KEY / GROQ_API_KEY (tier 2) or GEMINI_API_KEY (tier 3); whisper.cpp is then never invoked.

Configure

cp env.example .env
# edit .env — nothing is required for tier 1

See env.example for every variable — all optional (API keys and tuning). Tier 1 needs none.

Register in Claude Code

Add to your project .mcp.json (or global config) — see .mcp.json.example:

{
  "mcpServers": {
    "video-vision": {
      "command": "uvx",
      "args": ["video-vision-mcp"],
      "env": { "VIDEO_MCP_ENV": "/abs/path/to/.env" }
    }
  }
}

uvx downloads and runs the published package automatically — no manual install step. VIDEO_MCP_ENV is optional (tier 1 needs no keys); point it at your .env if you use the cloud backends. For local development against a checkout, use "args": ["--from", "/abs/path/to/video-vision-mcp", "video-vision-mcp"] instead. Restart Claude Code; the video-vision tools then appear.

Cache

Results are cached at ~/.cache/video-vision-mcp/ keyed by (file hash, backend, frame interval) — re-analyzing the same video is instant, and switching backends or intervals keeps each result separately. Downloaded URLs and whisper models live under the same dir. Override with VIDEO_MCP_CACHE_DIR.

Cached analyses and downloaded videos older than VIDEO_MCP_CACHE_TTL_HOURS (default 24) are pruned on startup and skipped on read; set 0 to keep them forever. Whisper models are never pruned (expensive to re-download).

Using it with an integration (e.g. Jira, Slack)

This server is deliberately standalone — it never talks to Jira, Slack, or any other service. When a video lives behind an integration, let that integration's MCP fetch it, then pass the result here:

  1. The integration MCP downloads the attachment to a local file (or gives a direct, publicly reachable URL — an authenticated API URL won't work with url).

  2. Call analyze_video file_path=<downloaded file> (or url=<direct link>).

This keeps auth and service-specific logic where it belongs, and lets one video tool serve every source.

Available Tools

4 tools
analyze_videoA

Analyze a video into frames + transcript + metadata.

Provide exactly ONE source:

  • file_path: local path to a video file (already on disk).

  • url: direct/streaming URL (yt-dlp for known sites, HTTP otherwise).

frame_interval: seconds between sampled frames (default 1.0 = one per second). Denser sampling: 0.5 / 0.25 / 0.1; sparser: 2 / 5; or any custom value. The total is capped by the frame budget so long/dense videos can't flood context. Ignored by the native Gemini backend (it ingests the whole video).

The backend (local whisper.cpp / OpenAI / Groq / native Gemini) is chosen automatically from configured keys and named in the result metadata. Results are cached per (file-hash, backend, frame_interval); pass force_refresh=true to recompute.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNo
file_pathNo
force_refreshNo
frame_intervalNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description carries full burden. Covers automatic backend selection, caching, frame budget cap, and that frame_interval is ignored by Gemini. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

About 120 words, front-loaded purpose, clear sectioning for each parameter. No fluff, every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given output schema exists, description needn't detail return values. Covers inputs, parameters, caching, backend, and force_refresh. Complete for a tool with 4 params.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description must compensate. It explains each parameter: url/file_path are mutually exclusive, frame_interval possible values and behavior, force_refresh bypasses cache. Adds significant meaning beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'analyze', resource 'video', and outputs 'frames + transcript + metadata'. Differentiates from siblings like extract_frames_at and get_video_transcript_only by being comprehensive.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly requires exactly one source (file_path or url), provides guidance on frame_interval values, and mentions caching and force_refresh. Does not explicitly contrast with siblings but context implies this is the all-in-one tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_frames_atA

Extract frames at specific timestamps.

timestamps accepts seconds ("12", "12.5") or "MM:SS" / "HH:MM:SS". Tier 1/2 return real frame images; with the Gemini backend you get a textual description of those moments instead (no local frame cutting).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNo
file_pathNo
timestampsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses important behavioral traits: backend-dependent output (real images vs textual descriptions) and timestamp format support. This adds significant value beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two paragraphs) and front-loaded with the core purpose. Every sentence provides useful information, though the backend distinction could be more prominent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given three parameters and an output schema, the description covers timestamp format and backend behavior but omits guidance on selecting url vs file_path and does not mention output schema contents.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so description must compensate. It explains timestamps format well but does not clarify the roles of url and file_path parameters, leaving ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts frames at specific timestamps. It distinguishes from siblings like analyze_video, get_video_transcript_only, and list_recent_analyses, which have different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context on timestamp format and backend differences (real images vs textual descriptions), but does not explicitly state when to use this tool versus alternatives or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_video_transcript_onlyC

Fast path: return only the transcript text (no frame images).

Same inputs and backend selection as analyze_video. With the Gemini backend, returns Gemini's analysis text instead of a plain transcript.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNo
file_pathNo
force_refreshNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, description carries full burden. Only discloses that Gemini backend changes output format; no mention of auth, rate limits, mutability, or other behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Very concise (3 sentences), but lacks structure. Some essential info missing; could be better organized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 0% schema coverage and no annotation, the description is too sparse. Does not explain backend selection, parameter usage, or differentiate from siblings adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, yet description provides no parameter details. Only says 'same inputs as analyze_video', which is insufficient for an agent to understand parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it returns only transcript text (no frame images), and distinguishes from sibling tools like analyze_video. The mention of Gemini backend returning analysis text adds specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implied usage as a faster alternative to analyze_video for text-only needs, but lacks explicit when-to-use or when-not-to-use guidance. Does not mention alternatives or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_recent_analysesA

List previously analyzed videos from the cache, with the backend used for each.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It implies a read operation (listing cached items) but does not disclose any potential side effects, rate limits, or refresh behavior beyond the generic description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded, every word adds value. No unnecessary content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list tool with an output schema, the description sufficiently explains what the tool does and what extra info it provides (backend). No missing crucial details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, so schema coverage is trivially 100%. Baseline for 0 parameters is 4, and the description adds no parameter-specific info, which is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists previously analyzed videos from the cache and includes backend info. It distinguishes from sibling tools which perform analysis, frame extraction, or transcript retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool vs alternatives. While context implies it is for retrieving past analyses, no exclusions or alternative suggestions are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A3.8/5.0
Disambiguation4/5

Tools are generally distinct: analyze_video does full analysis, extract_frames_at targets specific timestamps, get_video_transcript_only is a fast path for transcript, and list_recent_analyses lists cache. However, get_video_transcript_only is a subset of analyze_video, which might cause some confusion for agents seeking only a transcript.

Naming Consistency5/5

All tool names use a consistent snake_case verb_noun pattern (analyze_video, extract_frames_at, get_video_transcript_only, list_recent_analyses), making them predictable and easy to understand.

Tool Count5/5

With 4 tools, the set is well-scoped for a video analysis server. Each tool serves a clear purpose without redundancy, and the number is neither too sparse nor excessive.

Completeness4/5

The tool surface covers core video analysis workflows: full analysis, targeted frame extraction, transcript retrieval, and cache listing. Minor gaps include lack of a delete cache tool or explicit backend configuration, but the set is functional for common use cases.

Maintenance

ActivityNo data
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/KitDevUA/video-vision-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server