# video-mcp

An MCP server that turns a script into a narrated video. Point an AI agent at it and it can
produce YouTube explainers (16:9) or Reels/Shorts (9:16) from three kinds of source:

| Engine | Tool | You supply | Good for |
|--------|------|------------|----------|
| **HTML scroll** | `generate_video` | An HTML page + segments naming a CSS selector each | Long-form explainers, docs walkthroughs, architecture pages |
| **Remotion** | `generate_video_remotion` | JSON props for a built-in template, or your own React component | Slide-style reels, data callouts, anything you'd design in React |
| **HyperFrames** | `generate_video_hyperframes` | A folder with `content.json` + `compose.mjs` (HTML + GSAP) | Kinetic-caption reels, motion-graphics styles you iterate on as code |

The idea shared by all three: **narration is generated first, and its real duration drives the
visuals.** Each segment gets the exact on-screen time its voice-over takes, so timing never drifts.
Captions come from the same timestamps for free.

## Requirements

- Node 20+
- `ffmpeg` and `ffprobe` on `PATH`
- An [ElevenLabs](https://elevenlabs.io) API key for narration
- Chromium for Playwright (`npx playwright install chromium`; Remotion downloads its own)

## Quickstart

```bash
git clone https://github.com/techbysaurabh/video-mcp && cd video-mcp
npm install && npx playwright install chromium
cp .env.example .env            # add ELEVENLABS_API_KEY
npm run demo:html               # silent render of the example page, no keys needed
npm start                       # MCP over HTTP at http://127.0.0.1:3100/mcp
```

Or with Docker (Chromium and ffmpeg included):

```bash
docker build -t video-mcp . && docker run -p 3100:3100 -e ELEVENLABS_API_KEY=... -v $PWD/output:/data/output video-mcp
```

### Connect a client

Claude Code, HTTP:

```bash
claude mcp add --transport http video-mcp http://127.0.0.1:3100/mcp
```

Claude Code, stdio (the client spawns the server; no port):

```bash
claude mcp add video-mcp -e ELEVENLABS_API_KEY=... -- node /path/to/video-mcp/src/mcp/server.js --stdio
```

Cursor (`.cursor/mcp.json`):

```json
{ "mcpServers": { "video-mcp": { "url": "http://127.0.0.1:3100/mcp" } } }
```

### First video

Ask your agent:

> Use video-mcp to render the bundled example with `generate_video`, then poll `get_job_status`
> until it's done and tell me where the MP4 is.

That renders `examples/html-scroll/page.html` with the five segments in `segments.json`, and writes
`output/<job>/output.mp4` plus `captions.srt`.

## Tools

| Tool | Purpose |
|------|---------|
| `generate_video` | HTML scroll engine. `html_path`, `segments` (`{id, scrollSelector, narrationText, holdAfterMs}`), `layout` desktop/mobile |
| `generate_video_remotion` | Remotion engine. `composition` SegmentReel/SegmentVideo + `props`, or `jsx_code`; optional narration `segments` |
| `generate_video_hyperframes` | HyperFrames engine. `style_dir` containing `content.json` and `compose.mjs` |
| `preview_narration` | TTS only; returns each segment's duration so you can tune the script before rendering |
| `preview_frames` | Screenshot an HTML page at given timestamps to check scroll targets |
| `get_job_status` / `list_jobs` | Renders are async; poll here. Results carry `videoPath`, `srtPath` and, if S3 is configured, `videoUrl` |
| `list_remotion_templates` | Props accepted by the built-in compositions |

## How it works

```
segments ──▶ ElevenLabs TTS (cached by voice+text) ──▶ narration.mp3 + per-segment timestamps
                                                            │
        ┌───────────────────────────────────────────────────┼──────────────────────────────┐
        ▼                                                   ▼                              ▼
 HTML: Playwright scrolls to each             Remotion: props get startTime/     HyperFrames: compose() gets
 scrollSelector on the timestamps,            duration, React renders to MP4     timed segments, emits HTML+GSAP,
 screenshots at 24 fps                                                           hyperframes renders it
        └───────────────────────────────────────────────────┴──────────────────────────────┘
                                                            │
                                     ffmpeg mux (video + narration) + SRT ──▶ output/<job>/output.mp4
                                                            │
                                            optional S3 upload (AWS_S3_BUCKET set)
```

Details worth knowing:

- **TTS cache.** Audio is cached by `sha256(voiceId + text)`; unchanged segments cost nothing on
  re-render.
- **Scroll motion.** Between segments the page eases to the next target over 0.6 s and holds. The
  `mobile` layout renders a 390 px CSS viewport scaled to 1080×1920 so responsive breakpoints kick in.
- **Remotion custom mode.** `jsx_code` is written to a per-job entry and bundled; the component
  receives `{segments, timestamps, totalDuration, ...props}` with each segment's `startTime` and
  `duration` resolved.
- **HyperFrames styles.** `compose.mjs` exports `({ segments, totalDuration, media, narration }) =>
  html`. Segments arrive with timing filled in; `media[id]` describes files a segment referenced via
  `localMedia`. See `examples/hyperframes-kinetic` for a caption style that splits each sentence into
  beats weighted by length.
- **Muxing.** Remotion emits a silent AAC track; the mux maps `0:v:0` and `1:a:0` explicitly so the
  narration always wins.
- **Cost guard.** `cost_ceil_chars` (default 50 000) aborts a job before any TTS is spent.

## Configuration

| Variable | Default | Meaning |
|----------|---------|---------|
| `ELEVENLABS_API_KEY` | | Required for narration |
| `ELEVENLABS_VOICE_ID` | `21m00Tcm4TlvDq8ikWAM` | Default voice |
| `ELEVENLABS_STABILITY` / `ELEVENLABS_STYLE` | `0.5` / `0.3` | Delivery tuning |
| `PORT` / `HOST` | `3100` / `127.0.0.1` | HTTP mode bind |
| `VIDEO_MCP_OUTPUT_DIR` | `./output` | Where jobs write |
| `AWS_S3_BUCKET`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | | Optional upload; results then include `videoUrl` |
| `HYPERFRAMES_VERSION` | `0.7.109` | Pinned `hyperframes` CLI version used via `npx` |

## CLI helpers

```bash
npm run demo:html                 # silent scroll render of the example page (no keys)
npm run demo:remotion             # silent SegmentReel render (no keys)
npm run render:hyperframes -- examples/hyperframes-kinetic   # full narrated render
npm test
```

## Scope

This repo is the rendering core: narration, three engines, muxing, captions, optional upload,
exposed over MCP. Publishing to platforms, AI-generated imagery, avatars and brand kits are
deliberately out of scope; build them on top.

## License

MIT