EditVideo Agent

Senses, hands and composition skills that let an AI edit video. EditVideo Agent is an MCP server and CLI that gives Claude, Codex or any MCP client the ability to read footage and audio, edit a revision-locked timeline or the generated source code, render with Remotion or HyperFrames, and check the result with measurements, then iterate.

It is not a video editor for people and has no studio UI. The calling model makes the editorial and aesthetic decisions; the runtime executes them deterministically and returns evidence (frames, crops, audio, numbers). There is no hidden beauty score and no approval gate. Tiếng Việt: README.vi.md.

v0.8.0 + unreleased upgrades: 137 MCP tools · 95 runnable effects (each with a before/after preview strip in both engines) + 7 tested combos · 18 layer kinds · 17 explainer recipes + 4 title recipes · 16 Vietnamese-complete font families · 24 palettes · 17 style kits · 7 original LUTs plus a scene-referred colour graph.

What the AI gets

Token economy. Tool I/O is compacted at the protocol edge: compact JSON with rounded floats, slimmed schemas, no duplicate structured copy, catalogs that list one-line rows until an exact id is asked for, handles (track_id, manifests) instead of dumps, and image packets whose text carries only evidence ids. EDITVIDEO_TOOLSET=compact exposes 7 entry tools for small-context hosts. See docs/AI_ERGONOMICS.md.

Related MCP server: video-editor

Demo

Three short archive films built end to end by the system from public-domain material (720p previews; credits in docs/demo):

Quick start

Requirements: Python 3.11+, Node.js 22+, FFmpeg/FFprobe on PATH. Windows is the tested platform; setup.sh covers the core runtime on Linux/macOS (less tested). The full guide, with prerequisites, optional extras, client configuration and troubleshooting, is INSTALL.md (tiếng Việt); what the system is and its contracts: docs/SPEC.md.

./setup.ps1            # core: Python venv, engines (npm ci), writes .mcp.json for this clone
./setup.ps1 -Vision    # + SAM 2 / depth / LaMa (CUDA, ~2.5 GB)
./setup.ps1 -Temporal  # + ProPainter and local LTX footage
./setup.ps1 -Voice     # + offline Vietnamese narration: Piper (GPL-3.0, separate process), vi_VN voice and Whisper small for word timing (~580 MB)

Claude Code: open the folder; .mcp.json (written by setup from .mcp.example.json) registers the server, and the skill lives in .claude/skills/editvideo-agent. Codex: skill in .agents/skills/editvideo-agent; add the server:

[mcp_servers.editvideo]
command = '<repo>\.venv\Scripts\python.exe'
args = ['<repo>\run.py']

Start every session with editvideo_bootstrap(task) → editvideo_discover_tools(query, include_schema=true). The bootstrap reports the server version and tool count so a stale client connection is easy to spot. One-shot CLI: echo '{"tool":"editvideo_list_projects","arguments":{}}' | python run.py --cli.

Verify

python -m pip install -r requirements-dev.txt
python -m pytest -q

About 600 tests run in about 15 minutes on CPU (FFmpeg, Node and the Remotion Chromium are needed; GPU model tests only run with EDITVIDEO_RUN_RUNTIME_TESTS=1, and tests of optional extras skip when the extra is not installed). The suite uses its own temporary data store. Synthetic ground-truth generators (tests/synth_*.py) check music, dialogue and multicam analysis against known answers.

Repository layout

editvideo/        MCP server, timeline model, perception, recipes, colour, audio, review
engines/          Remotion entry (AgentVideo.tsx), HyperFrames runtime, shared effects.js / scene3d.js
assets/           bundled fonts (with licences), original LUTs, vendored library previews
.claude/ .agents/ skills for Claude Code and Codex; the independent reviewer agent
docs/             guides (workflows, colour/3D, explainer kit, review rubric, research log)
examples/         reference build scripts (the sample films)
scripts/          setup helpers, atlas builders, repo facts
tests/            pytest suite and synthetic ground-truth generators

Runtime state (data/) and renders (outputs/) stay local and are git-ignored; publish chosen renders as release assets.

Documentation

Install · Specification · Demo films · Pro workflows · Explainer kit · Colour, depth & 3D · Design library · Model integration · AI ergonomics · Review rubric · Learning from references · Changelog · Contributing · Code of conduct · Security

Licence

EditVideo Agent is licensed under the Apache License 2.0 (NOTICE). The engines, fonts, models, data and media it uses keep their own licences: see THIRD_PARTY_NOTICES.md.

• Remotion is free for individuals and small companies, but larger for-profit organisations need a Remotion company licence. HyperFrames (Apache-2.0) is the open engine.

• ProPainter is non-commercial.

• Piper (GPL-3.0) runs as a separate process.