# video-brief-mcp

An MCP server that turns a screen-recorded walkthrough — the "here's my feedback, let me show you" video a
client or PM sends — into something a coding assistant can actually read: a timestamped transcript, frames,
scene changes, and **slices** that pair what was said with what was on screen.

Everything runs locally: ffmpeg for video, [openai-whisper](https://github.com/openai/whisper) for speech.
Nothing is uploaded anywhere.

## Tools

| Tool | What it does |
| --- | --- |
| `brief_probe` | Duration, resolution, fps, audio, size, and whether a transcript is cached. Start here. |
| `brief_transcribe` | Runs Whisper as a background job. Waits up to `waitSeconds`, then returns the full transcript or progress (`03:30 of 13:13 (26%)`). Cached on disk, so repeat calls are instant. |
| `brief_transcript` | Read a time range of the cached transcript, or `search` for a phrase with context. |
| `brief_frames` | Frames as images at explicit times (`at: ["2:15", 250]`) or every N seconds; 16 per call, with a continuation hint. |
| `brief_scenes` | Moments the screen changes (page switch, modal), thinned to one per few seconds; optionally with frames. Better anchors than fixed intervals. |
| `brief_slice` | The main reading tool: transcript of `[from, to]` interleaved with frames, each frame followed by the speech that comes after it. |
| `brief_export` | Writes `transcript.txt`, `frames/frame_NNN.jpg` and a README skeleton into a folder you keep next to the task. Never overwrites an existing README. |

Times accept seconds, `MM:SS` or `HH:MM:SS`. All tools are read-only except `brief_export`.

## Requirements

- macOS or Linux with `ffmpeg` / `ffprobe` on PATH (`brew install ffmpeg`)
- Python 3 with `openai-whisper` (`pip install openai-whisper`); the first run of a model downloads it to `~/.cache/whisper`

## Setup

```sh
npm install && npm run build
claude mcp add video-brief --scope user -- node /absolute/path/to/video-brief-mcp/dist/index.js
```

Optional environment: `VIDEO_BRIEF_MODEL` (default `small`), `VIDEO_BRIEF_LANGUAGE` (default `en`, or `auto`),
`VIDEO_BRIEF_PYTHON`, `FFMPEG_PATH`, `FFPROBE_PATH`.

## A typical session

1. `brief_probe /path/review.mov` → 13:13, 2282×1316, audio
2. `brief_transcribe` → a minute later, the transcript
3. `brief_scenes withFrames: true` → where the screen changes, as images
4. For each item in the checklist: `brief_slice from: "2:15" to: "4:10"` → what she said and what she pointed at
5. `brief_export outDir: docs/review-oct12` → the folder that goes into the repo next to the task

On an M1 Pro with the `small` model, one minute of speech transcribes in about 8 seconds on CPU.

## Development

```sh
npm test          # vitest; ffmpeg and whisper are injected, no media needed
npm run typecheck
```

- `src/media.ts` — ffprobe / ffmpeg: probe, single-frame grabs (input seeking, fast on large files), scene detection
- `src/transcripts.ts` — background Whisper jobs, progress parsing, disk cache, slicing and search
- `scripts/transcribe.py` — Whisper runner that prints JSON and progress lines
- `src/tools.ts` — tool definitions and handlers
- `src/index.ts` — MCP wiring over stdio