Omni-Video Studio MCP
Provides high-fidelity word-level transcription mapping during video ingestion.
Provides advanced rendering pipeline with FFmpeg filter graphs for EDL cuts, overlay rendering, subtitle burning, LUT color grading, and audio restoration.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Omni-Video Studio MCP ingest the raw footage and create a 30-second preview"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Omni-Video Studio MCP
The Omni-Video Studio MCP is an enterprise-grade, autonomous Model Context Protocol (MCP) server that empowers any LLM-enabled IDE (Cursor, Claude Code, Antigravity) to act as a professional video editor.
It evolves simple transcription-based editing into a deterministic, token-efficient, and pipeline-driven workflow, featuring agent-native motion graphics (Hyperframes), visual metadata proxies, and high-fidelity final renders.
π Key Features
Metadata Proxy Ingestion: Instead of streaming expensive video tokens to an LLM, this server pre-processes footage to extract a
takes_packed.md(audio mapping) and a Visual Scene Graph. The agent edits using text proxies, cutting costs and accelerating reasoning.Hyperframes Engine: Forget complex Node.js dependencies (e.g., Remotion). The agent generates deterministic HTML/CSS motion graphics which are instantly rendered to transparent video using Playwright.
Advanced Rendering Pipeline: Powered by robust FFmpeg filter graphs, the final output supports EDL (Edit Decision List) cuts, overlay rendering, Subtitle burning, LUT color grading, and optional DeepFilterNet AI audio restoration.
IDE-Agnostic: Because it adheres to the official MCP specification, it drops directly into Cursor, Antigravity, or Claude Desktop without custom plugins.
Related MCP server: Video & Audio Editing MCP Server
π¦ Installation
Prerequisites:
python 3.10+ffmpeg(must be installed on your system path)uv(recommended for dependency management)
# Clone the repository
git clone https://github.com/Mallet-Builds/omni-video-mcp.git
cd omni-video-mcp
# Install dependencies
uv venv
source .venv/bin/activate
uv pip install -e .
# Install Playwright browsers (for Hyperframes)
playwright install chromiumπ Configuration
Add the server to your IDE's MCP settings file (e.g., ~/.gemini/antigravity/mcp_config.json, ~/.cursor/mcp.json, or Claude Desktop config):
{
"mcpServers": {
"omni-video-mcp": {
"command": "uv",
"args": [
"run",
"/path/to/omni-video-mcp/server.py"
],
"env": {
"ELEVENLABS_API_KEY": "your_api_key_here"
}
}
}
}Note: The ELEVENLABS_API_KEY is currently required for high-fidelity word-level transcription mapping during ingestion.
π¬ How it Works (The Agent Pipeline)
When the agent uses this MCP server, it follows a 4-phase architecture:
Phase 1: Ingestion (
omni_video_ingest) The agent scans your raw.mp4/.movfiles, extracting a packed markdown transcript and an initial Visual Scene Graph.Phase 2: Director's Cut (
omni_video_preview) The agent uses the transcript to construct an EDL (Edit Decision List) of the best takes. Ambiguous cuts can be visually verified by generating filmstrip PNGs via the preview tool.Phase 3: VFX (
omni_video_generate_vfx) The agent generates HTML/CSS motion graphics (lower thirds, b-roll layouts) and the server renders them deterministically into transparent.webmvideos via Hyperframes.Phase 4: Sweetening & Render (
omni_video_render) The agent passes the EDL, VFX timestamps, and render settings to the server, which builds a complex FFmpeg graph to concatenate the footage, grade it, restore the audio, and export a final master.
π€ Contributing
Contributions are welcome! If you're adding new render pipeline capabilities (like auto-tracking or local whisper fallbacks), please open a PR. Ensure that any added Python dependencies are added to the pyproject.toml using uv add <package>.
π License
MIT License
Available Tools
4 toolsomni_video_generate_vfxC
Renders motion graphics (e.g., lower thirds, titles) using Hyperframes. Returns the path to the rendered transparent .mov or .webm file.
Inputs: request (required). Returns: result
omni_video_ingestA
Ingests a directory of video files, generates word-level audio transcripts, and constructs a semantic Visual Scene Graph for B-Roll searching. Returns the path to the generated project metadata.
Inputs: request (required). Returns: result
omni_video_previewA
Generates a filmstrip PNG of the specified video segment. Useful for visually verifying cut boundaries or B-roll placement. Returns the absolute path to the generated PNG file.
Inputs: request (required). Returns: result
omni_video_renderA
Orchestrates the final render pipeline: applies EDL cuts, overlay graphics, LUT color grading, audio restoration, and subtitle burning. Returns the path to the final rendered video.
Inputs: request (required). Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogβ¦
TDQS
Scored across 4 tools
Each tool serves a distinct purpose: ingest (import & metadata), preview (filmstrip generation), generate_vfx (motion graphics), and render (final pipeline). No overlap.
All tools follow 'omni_video_verb' pattern consistently: generate_vfx, ingest, preview, render. Uniform and predictable.
4 tools cover the core video workflow (ingest, preview, VFX, render) without unnecessary complexity. Well-scoped.
Covers key stages of video production, but could include additional operations like asset management or format export. Minor gap.
Related MCP Connectors
AI video editor your agent can drive: plan scenes, edit a multi-track timeline, and render MP4s.
Open-source AI video editor: cut, caption, clip and animate real Revline projects from your agent.
FFmpeg as a service for AI agents: typed video editing tools, async jobs, downloadable outputs.
The agentic video editor for ChatGPT and Claude. Ask in your chat and it cuts, captions, adds music, titles and motion graphics on a real timeline you can open and adjust.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables video editing using natural language commands powered by FFmpeg, supporting operations like trimming, merging, format conversion, and more with real-time progress tracking and error handling.52-
- AlicenseBqualityDmaintenanceProvides powerful video and audio editing capabilities through FFmpeg, enabling AI assistants to perform professional-grade operations including format conversion, trimming, overlays, transitions, and advanced audio processing.2790MIT
- FlicenseAqualityDmaintenanceAn MCP server that transforms LLM-enabled IDEs into professional video editors by pre-processing footage into text proxies, generating motion graphics via HTML/CSS, and orchestrating complex FFmpeg renders.4-
- FlicenseAqualityBmaintenanceEnables AI-powered video editing in IDEs through a pipeline-driven workflow, including proxy ingestion, motion graphics generation, and high-fidelity rendering.4-