🎬 Video Reader MCP

Your agent watched the video. Did it read the timeline?

Evidence-first video reading for AI agents. One call turns any local video into a timeline document — ffprobe streams, chapters, embedded subtitles, scene boundaries, and warnings you can cite without frame-by-frame vision LLM calls.

Local-first · One smart read_video call · Timeline evidence + provenance · 20 tests

SOTA family roadmap: docs/roadmap/sota-family-roadmap.md.

⭐ Star this repo if agents should read video timelines with proof, not sampled frame captions. · Quick start · See it work · Why not frame-by-frame vision?

Part of the Sylphx Reader portfolio — orchestration and portfolio ADR live in smart-reader-mcp.

The problem

Videos are not a pile of frames. They are codecs, chapters, embedded subtitles, scene cuts, variable frame rates, and timelines measured in milliseconds.

Most agent stacks sample frames and ask a vision LLM what it sees. Subtitles get skipped. Scene boundaries vanish. Duration and stream metadata never reach context. Citations become "around minute two, I think." Then the agent hallucinates — confidently.

Video Reader MCP is built for the moment your agent needs a citeable timeline, not a slideshow summary.

Related MCP server: vidsight

Why not frame-by-frame vision?

See it work

Install once. Call once.

claude mcp add video-reader -- npx @sylphx/video-reader-mcp

{
  "sources": [{ "path": "/absolute/path/to/demo.mp4" }],
  "include_subtitles": true,
  "include_scenes": true
}

read_video builds a timeline document per source — no per-frame vision LLM calls:

{
  "source": "/absolute/path/to/demo.mp4",
  "success": true,
  "data": {
    "provenance": {
      "source": "/absolute/path/to/demo.mp4",
      "tool": "read_video",
      "version": "0.1.0",
      "extracted_at": "2026-07-09T12:00:00.000Z"
    },
    "format": {
      "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
      "duration_ms": 125500
    },
    "streams": [
      { "index": 0, "codec_type": "video", "width": 1920, "height": 1080 },
      { "index": 1, "codec_type": "audio", "channels": 2, "sample_rate": 48000 }
    ],
    "chapters": [
      { "id": 0, "start_ms": 0, "end_ms": 60250, "title": "Intro" }
    ],
    "subtitles": [
      {
        "index": 0,
        "start_ms": 1200,
        "end_ms": 3400,
        "text": "Welcome to the demo.",
        "provenance": { "method": "ffmpeg_extract", "format": "srt" }
      }
    ],
    "scenes": [
      {
        "index": 0,
        "time_ms": 45200,
        "provenance": { "method": "ffmpeg_scene_filter", "threshold": 0.4 }
      }
    ],
    "warnings": []
  }
}

Abbreviated shape — optional local ASR transcript hooks skip gracefully when no adapter is wired.

Prerequisites

• Node.js >=22.13

• ffprobe (required) and ffmpeg (recommended for subtitles + scenes) on PATH

MCP Tool Surface

Supported formats: MP4, M4V, MKV, MOV, WebM, and other formats ffprobe can inspect.

Quick Start

Claude Code

claude mcp add video-reader -- npx @sylphx/video-reader-mcp

Claude Desktop

Add this to claude_desktop_config.json:

{
  "mcpServers": {
    "video-reader": {
      "command": "npx",
      "args": ["@sylphx/video-reader-mcp"]
    }
  }
}

Any MCP Client

npx @sylphx/video-reader-mcp

HTTP transport (optional)

MCP_TRANSPORT=http MCP_HTTP_PORT=8080 npx @sylphx/video-reader-mcp

Security model

• Local-first — read_video inspects local files; remote URLs are not fetched by default.

• ffprobe/ffmpeg boundary — probe and frame tools shell out to configured binaries on PATH; missing tools return explicit errors.

• Fixture corpus — CI validates parser and safety fixtures; corrupted inputs fail closed with structured diagnostics.

• Evidence envelope — timestamps, frame indices, and extraction routes are preserved so agents can verify claims.

Release proof

Claims are backed by CI benchmark:release-gate, fixture corpus checks, and the shipped-path matrix (Rust-default primary tools).

bun run benchmark:release-gate

Artifact: benchmark-artifacts/video_reader_release_gate.json — must report status: passed before release.

Development

git clone https://github.com/SylphxAI/video-reader-mcp.git
cd video-reader-mcp
bun install
bun run build
bun test
bun run doctor
bun run benchmark:release-gate

Useful checks:

bun run check
bun run typecheck
bun run benchmark:release-gate

Example read_video requests live in examples/. CI runs parser, fixture corpus, doctor, and release-gate checks; integration tests exercise ffmpeg when available on the runner.

Support

• Issues

• npm package

• Portfolio orchestration: smart-reader-mcp

Help this reach more builders

If frame-by-frame vision guesses have wasted your context, your citations, or your trust in agent output, you are exactly who this project is for.

⭐ Star the repo — it is the fastest way to help more agent builders find evidence-first video reading. Share it in your MCP client setup, team wiki, or agent stack README.

Discovery (in progress)

Know another MCP directory? Open an issue with the link.

License

MIT © SylphxAI