Cue

Timeline proof for agents — not frame-by-frame vision guesses.

Local-first video evidence: streams, chapters, subtitles, scenes, and follow-up crops your agent can cite.

Canonical @sylphx/cue · bin cue · live 0.2.1

Zero-config in one line

npx -y @sylphx/cue

No API key. No global install. Starts a stdio MCP server immediately.

Related MCP server: Video MCP Server

Why Cue feels unfairly good

Your agent watched the video. Did it read the timeline?

Five reasons teams pick Cue

1. Zero-config MCP for video evidence.

2. Timeline twin, not random frame captions.

3. Local-first — no required upload of whole videos to a cloud API.

4. Fail closed without the matching native.

5. Family ready — Iris for stills, Citra for PDFs, Locus for code.

What agents get

Public tools: read_video, video_evidence.

Flagship use cases

1. Meeting / lecture recordings — chapters, subtitles, scene boundaries

2. Product demos — extract citeable frames with geometry context

3. Long-form media triage — structure first, then crop evidence

Product docs

See it work

Install (30 seconds)

npm install -g @sylphx/cue
cue doctor
claude mcp add cue -- npx -y @sylphx/cue

Install once. Call once.

{
  "sources": [{ "path": "/absolute/path/to/demo.mp4" }],
  "include_subtitles": true,
  "include_scenes": true
}

read_video builds a timeline document per source — no per-frame vision LLM calls:

{
  "source": "/absolute/path/to/demo.mp4",
  "success": true,
  "data": {
    "provenance": {
      "source": "/absolute/path/to/demo.mp4",
      "tool": "read_video",
      "version": "0.1.0",
      "extracted_at": "2026-07-09T12:00:00.000Z"
    },
    "format": {
      "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
      "duration_ms": 125500
    },
    "streams": [
      { "index": 0, "codec_type": "video", "width": 1920, "height": 1080 },
      { "index": 1, "codec_type": "audio", "channels": 2, "sample_rate": 48000 }
    ],
    "chapters": [
      { "id": 0, "start_ms": 0, "end_ms": 60250, "title": "Intro" }
    ],
    "subtitles": [
      {
        "index": 0,
        "start_ms": 1200,
        "end_ms": 3400,
        "text": "Welcome to the demo.",
        "provenance": { "method": "ffmpeg_extract", "format": "srt" }
      }
    ],
    "scenes": [
      {
        "index": 0,
        "time_ms": 45200,
        "provenance": { "method": "ffmpeg_scene_filter", "threshold": 0.4 }
      }
    ],
    "warnings": []
  }
}

Abbreviated shape — optional local ASR transcript hooks skip gracefully when no adapter is wired.

Prerequisites

• Node.js >=22.13

• ffprobe (required) and ffmpeg (recommended for subtitles + scenes) on PATH

MCP Tool Surface

Supported formats: MP4, M4V, MKV, MOV, WebM, and other formats ffprobe can inspect.

Quick Start

Claude Code

Claude Desktop

Add this to claude_desktop_config.json:

{
  "mcpServers": {
    "cue": {
      "command": "npx",
      "args": ["-y", "@sylphx/cue"]
    }
  }
}

Any MCP Client

npx -y @sylphx/cue

HTTP transport (optional)

MCP_TRANSPORT=http MCP_HTTP_PORT=8080 npx -y @sylphx/cue

Security model

• Local-first — read_video inspects local files; remote URLs are not fetched by default.

• ffprobe/ffmpeg boundary — probe and frame tools shell out to configured binaries on PATH; missing tools return explicit errors.

• Fixture corpus — CI validates parser and safety fixtures; corrupted inputs fail closed with structured diagnostics.

• Evidence envelope — timestamps, frame indices, and extraction routes are preserved so agents can verify claims.

Release proof

Claims are backed by CI benchmark:release-gate, fixture corpus checks, and the shipped-path matrix (Rust-default primary tools).

bun run benchmark:release-gate

Artifact: benchmark-artifacts/video_reader_release_gate.json — must report status: passed before release.

Development

git clone https://github.com/SylphxAI/video-reader-mcp.git
cd video-reader-mcp
bun install
bun run build
bun test
bun run doctor
bun run benchmark:release-gate

Useful checks:

bun run check
bun run typecheck
bun run benchmark:release-gate

Example read_video requests live in examples/. CI runs parser, fixture corpus, doctor, and release-gate checks; integration tests exercise ffmpeg when available on the runner.

Support

• Issues

• npm package

• Portfolio orchestration: smart-reader-mcp

Help this reach more builders

If frame-by-frame vision guesses have wasted your context, your citations, or your trust in agent output, you are exactly who this project is for.

⭐ Star the repo — it is the fastest way to help more agent builders find evidence-first video reading. Share it in your MCP client setup, team wiki, or agent stack README.

Discovery (in progress)

Know another MCP directory? Open an issue with the link.

License

MIT © SylphxAI