ffmpeg-mcp-video-editor

Features

• Typed tools, not a shell — ffmpeg's editing surface arrives as 38 schema'd MCP tools, so the calling model gets parameters and guardrails instead of hand-writing filter graphs.

• Nothing blocks — anything that encodes frames returns a job id immediately, with real progress parsed from ffmpeg's own output and a cancel that actually kills the subprocess.

• Zero setup — finds your ffmpeg, or downloads a verified static build for your OS on first run. The Whisper and face models fetch themselves too.

• Captions that survive their own text — a subtitle containing Time: 12:30, [note]; it's 50% would corrupt a naively built filter graph. Here it doesn't, and there's a test proving it.

• Sees faces — detect them, follow one with a smoothed crop to turn landscape into vertical, or blur every face except your subject.

• Hears speech — Whisper transcription, translation, and one-shot auto-captioning with word-level timestamps.

• Renders whole timelines — clips, transitions, overlays, captions and ducked audio tracks compile into a single ffmpeg pass.

• Two front doors, one backend — the optional local UI shares the same job store, so it and your MCP client see and cancel each other's work.

• Can see what it edits — extract a frame or a contact sheet, measure brightness, colour and loudness. Editing is a loop, so the server can check its own output rather than rendering blind.

• Tested where it counts — 727 tests; the unit tests run with no ffmpeg installed at all.

Related MCP server: ffmpeg-mcp

Quick start

uv sync --all-extras          # core + Whisper + vision + UI
uv run ffmpeg-mcp-server

Register it with your MCP client. Rather than hand-editing paths, print the block with your real ones already filled in:

printf '{\n  "mcpServers": {\n    "ffmpeg-mcp": {\n      "command": "%s",\n      "args": ["run", "--directory", "%s", "ffmpeg-mcp-server"]\n    }\n  }\n}\n' "$(command -v uv)" "$PWD"

Paste the result into your client's MCP config, then restart it. Both paths must be absolute: a GUI app does not inherit your shell's PATH, and it launches the server from an unrelated working directory.

That is uv failing before the server ever starts, and it almost always means the --directory path does not exist — most often a placeholder that was pasted verbatim. Run the command above to get the correct one. You can check it directly with:

uv run --directory /your/path ffmpeg-mcp-server   # should print startup logs, not exit

If uv itself is not found, use its absolute path (command -v uv) as "command".

Then just ask for what you want:

"Take interview.mp4, follow the speaker's face, make it a vertical Reel, and burn in captions."

track_and_crop  →  job 8f3a…  ▸ done   interview_reframed.mp4
auto_caption    →  job b71c…  ▸ done   interview_reframed_captioned.mp4  + .srt

probe_media and list_capabilities answer instantly. Everything else returns a job id — poll job_status, then job_result. cancel_job stops it mid-render.

Give it real paths on your machine. The server runs natively, so it reads your filesystem — not files you attach to the chat, which live in the assistant's own sandbox. Say ~/Downloads/clip.mov, not /mnt/user-data/uploads/clip.mov. Paths must also sit under FFMPEG_MCP_ALLOWED_ROOTS, which defaults to your home directory.

Projects

Several sessions can share one server without mixing together. Name a project and everything that session does is filed under it:

set_project { "name": "dress-reel" }        // one session
set_project { "name": "wedding-teaser" }    // another, same server

Jobs are stamped with the project, unspecified outputs land in <workspace>/projects/<name>/, and list_jobs shows only that project by default — so a busy shared queue stays readable. project: "all" spans them, and list_projects shows what exists with per-project counts.

Long queues page rather than dumping everything:

list_jobs { "limit": 25, "offset": 50 }     // returns total and has_more

Workers still share the queue, so projects divide the bookkeeping without dividing the compute.

How it works

Tools validate their arguments and enqueue; a worker pool picks the job up and runs it. The job store is plain SQLite inside the workspace, which is what lets the MCP server and the UI be two processes over one queue — either can enqueue, either can watch progress, and either can cancel a job the other one started, because cancellation is a cooperative flag the owning worker polls.

Every ffmpeg invocation goes through one async execution path that owns timeouts, progress parsing and error wrapping. Arguments are always a list; shell=True appears nowhere in the project.

Reframing for Reels and Shorts

The most-asked-for edit, as one call. resize_video takes a named preset (19 of them), an aspect_ratio like 9:16, or explicit dimensions — and fit decides what happens to the picture that no longer fits.

{ "input_path": "talk.mp4", "preset": "reel",   "fit": "blur"  }   // blurred bars, nothing cropped
{ "input_path": "talk.mp4", "aspect_ratio": "1:1", "focus": "top" } // crop, keep the top

For a talking head that must stay in frame, prefer track_and_crop — it follows the face and smooths the crop path, because a crop that snaps frame to frame looks worse than a slightly imperfect one that glides.

The tools

Six tools are read-only and answer synchronously — probe_media, list_capabilities, list_resolution_presets, and the three job queries. The rest return a job id.

The local UI

A separate process over the same workspace and job store — run it alongside the MCP server, or entirely on its own.

uv run ffmpeg-mcp-ui          # http://127.0.0.1:8756

Dark, glassmorphic, one orange accent. Built assets are committed, so running the UI needs no Node toolchain.

Requirements

• Python 3.11 or 3.12 (managed with uv)

• ffmpeg 6+ — or let it download a static build on first run

• macOS, Linux, or Windows

Configuration

Environment variables, all prefixed FFMPEG_MCP_:

How ffmpeg is found

Configured path → build cached in the workspace → a system ffmpeg ≥ 6 → a downloaded static build.

Builds from BtbN (Linux, Windows) are verified against the checksums.sha256 published with the release, and a mismatch is fatal. evermeet.cx (macOS) publishes only a GPG signature and no digest, so macOS downloads are pinned on first use — the hash is recorded in <workspace>/bin/manifest.json and any later download of the same URL must match. Prefer to avoid that? Install ffmpeg yourself and it'll be used instead.

The MediaPipe face model (~230 KB) is fetched on first vision call and verified against a pinned SHA-256.

Security

• Path allowlist — every input and output is resolved through symlinks before being checked, so a symlink in the workspace can't reach /etc.

• No shell, ever — ffmpeg arguments are always a list. shell=True appears nowhere.

• Filter escaping, verified — filter strings are built in one module applying both levels of ffmpeg's escaping. Overlay text goes to a sidecar file referenced with textfile= and expansion=none, so caption text never enters the graph at all.

• Auditable — every job records its resolved ffmpeg command line. File contents are never logged.

• The UI is the same trust boundary — on loopback it runs unauthenticated; bound anywhere else it generates and requires a token.

• Uploads are rebuilt, not filtered — the directory component is discarded (../../etc/passwd → passwd), the stem reduced to [A-Za-z0-9._-], and the extension must be a media type the tools read.

Development

uv run pytest                       # 727 tests
uv run pytest -m "not integration"  # most need no ffmpeg
uv run ruff check . && uv run ruff format --check .
uv run mypy

Unit tests cover the pure logic — filter builders and escaping, path validation, the job store, SRT and LUT parsing, tracking geometry, timeline compilation. Integration tests generate small fixture clips on first run and verify rendered output by probing it.

Rebuilding the frontend:

cd ui-src && npm install && npm run build   # emits into src/ffmpeg_mcp/ui/static/

Notes

blur_faces covers up to 12 tracked faces and says so when it truncates. A missed detection means an unblurred face — review the output before publishing anything sensitive.

License

MIT © AbyAbyss.

ffmpeg itself is separately licensed, and the static builds this server can download for you are GPL-configured. Those run as a separate process that this project invokes over a command line — it links no ffmpeg code — so the MIT terms above cover this codebase only. If you redistribute a bundle that ships an ffmpeg binary alongside it, check that binary's own terms. Point FFMPEG_MCP_FFMPEG_PATH at an LGPL build if you would rather avoid GPL components entirely.