Skip to main content
Glama

Omni-Video Studio MCP

The Omni-Video Studio MCP is an enterprise-grade, autonomous Model Context Protocol (MCP) server that empowers any LLM-enabled IDE (Cursor, Claude Code, Antigravity) to act as a professional video editor.

It evolves simple transcription-based editing into a deterministic, token-efficient, and pipeline-driven workflow, featuring agent-native motion graphics (Hyperframes), visual metadata proxies, and high-fidelity final renders.

🌟 Key Features

  1. Metadata Proxy Ingestion: Instead of streaming expensive video tokens to an LLM, this server pre-processes footage to extract a takes_packed.md (audio mapping) and a Visual Scene Graph. The agent edits using text proxies, cutting costs and accelerating reasoning.

  2. Hyperframes Engine: Forget complex Node.js dependencies (e.g., Remotion). The agent generates deterministic HTML/CSS motion graphics which are instantly rendered to transparent video using Playwright.

  3. Advanced Rendering Pipeline: Powered by robust FFmpeg filter graphs, the final output supports EDL (Edit Decision List) cuts, overlay rendering, Subtitle burning, LUT color grading, and optional DeepFilterNet AI audio restoration.

  4. IDE-Agnostic: Because it adheres to the official MCP specification, it drops directly into Cursor, Antigravity, or Claude Desktop without custom plugins.

Related MCP server: Video & Audio Editing MCP Server

πŸ“¦ Installation

Prerequisites:

  • python 3.10+

  • ffmpeg (must be installed on your system path)

  • uv (recommended for dependency management)

# Clone the repository
git clone https://github.com/Mallet-Builds/omni-video-mcp.git
cd omni-video-mcp

# Install dependencies
uv venv
source .venv/bin/activate
uv pip install -e .

# Install Playwright browsers (for Hyperframes)
playwright install chromium

πŸ›  Configuration

Add the server to your IDE's MCP settings file (e.g., ~/.gemini/antigravity/mcp_config.json, ~/.cursor/mcp.json, or Claude Desktop config):

{
  "mcpServers": {
    "omni-video-mcp": {
      "command": "uv",
      "args": [
        "run",
        "/path/to/omni-video-mcp/server.py"
      ],
      "env": {
        "ELEVENLABS_API_KEY": "your_api_key_here" 
      }
    }
  }
}

Note: The ELEVENLABS_API_KEY is currently required for high-fidelity word-level transcription mapping during ingestion.

🎬 How it Works (The Agent Pipeline)

When the agent uses this MCP server, it follows a 4-phase architecture:

  1. Phase 1: Ingestion (omni_video_ingest) The agent scans your raw .mp4 / .mov files, extracting a packed markdown transcript and an initial Visual Scene Graph.

  2. Phase 2: Director's Cut (omni_video_preview) The agent uses the transcript to construct an EDL (Edit Decision List) of the best takes. Ambiguous cuts can be visually verified by generating filmstrip PNGs via the preview tool.

  3. Phase 3: VFX (omni_video_generate_vfx) The agent generates HTML/CSS motion graphics (lower thirds, b-roll layouts) and the server renders them deterministically into transparent .webm videos via Hyperframes.

  4. Phase 4: Sweetening & Render (omni_video_render) The agent passes the EDL, VFX timestamps, and render settings to the server, which builds a complex FFmpeg graph to concatenate the footage, grade it, restore the audio, and export a final master.

🀝 Contributing

Contributions are welcome! If you're adding new render pipeline capabilities (like auto-tracking or local whisper fallbacks), please open a PR. Ensure that any added Python dependencies are added to the pyproject.toml using uv add <package>.

πŸ“„ License

MIT License

Available Tools

4 tools
omni_video_generate_vfxC

Renders motion graphics (e.g., lower thirds, titles) using Hyperframes. Returns the path to the rendered transparent .mov or .webm file.

Inputs: request (required). Returns: result

omni_video_ingestA

Ingests a directory of video files, generates word-level audio transcripts, and constructs a semantic Visual Scene Graph for B-Roll searching. Returns the path to the generated project metadata.

Inputs: request (required). Returns: result

omni_video_previewA

Generates a filmstrip PNG of the specified video segment. Useful for visually verifying cut boundaries or B-roll placement. Returns the absolute path to the generated PNG file.

Inputs: request (required). Returns: result

omni_video_renderA

Orchestrates the final render pipeline: applies EDL cuts, overlay graphics, LUT color grading, audio restoration, and subtitle burning. Returns the path to the final rendered video.

Inputs: request (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 4 tools

    Disambiguation5/5

    Each tool serves a distinct purpose: ingest (import & metadata), preview (filmstrip generation), generate_vfx (motion graphics), and render (final pipeline). No overlap.

    Naming Consistency5/5

    All tools follow 'omni_video_verb' pattern consistently: generate_vfx, ingest, preview, render. Uniform and predictable.

    Tool Count5/5

    4 tools cover the core video workflow (ingest, preview, VFX, render) without unnecessary complexity. Well-scoped.

    Completeness4/5

    Covers key stages of video production, but could include additional operations like asset management or format export. Minor gap.

    Related MCP Connectors

    Related MCP Servers