Provides high-fidelity word-level transcription mapping during video ingestion via ElevenLabs API.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Omni-Video Studio MCP ingest my raw footage and create a preview EDL"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
The Omni-Video Studio MCP is an enterprise-grade, autonomous Model Context Protocol (MCP) server that empowers any LLM-enabled IDE (Cursor, Claude Code, Antigravity) to act as a professional video editor.
It evolves simple transcription-based editing into a deterministic, token-efficient, and pipeline-driven workflow, featuring agent-native motion graphics (Hyperframes), visual metadata proxies, and high-fidelity final renders.
Metadata Proxy Ingestion: Instead of streaming expensive video tokens to an LLM, this server pre-processes footage to extract a takes_packed.md (audio mapping) and a Visual Scene Graph. The agent edits using text proxies, cutting costs and accelerating reasoning.
Hyperframes Engine: Forget complex Node.js dependencies (e.g., Remotion). The agent generates deterministic HTML/CSS motion graphics which are instantly rendered to transparent video using Playwright.
Advanced Rendering Pipeline: Powered by robust FFmpeg filter graphs, the final output supports EDL (Edit Decision List) cuts, overlay rendering, Subtitle burning, LUT color grading, and optional DeepFilterNet AI audio restoration.
IDE-Agnostic: Because it adheres to the official MCP specification, it drops directly into Cursor, Antigravity, or Claude Desktop without custom plugins.
Related MCP server: omni-video-mcp
Prerequisites:
python 3.10+
ffmpeg (must be installed on your system path)
uv (recommended for dependency management)
# Clone the repository
git clone https://github.com/Mallet-Builds/omni-video-mcp.git
cd omni-video-mcp
# Install dependencies
uv venv
source .venv/bin/activate
uv pip install -e .
# Install Playwright browsers (for Hyperframes)
playwright install chromiumAdd the server to your IDE's MCP settings file (e.g., ~/.gemini/antigravity/mcp_config.json, ~/.cursor/mcp.json, or Claude Desktop config):
{
"mcpServers": {
"omni-video-mcp": {
"command": "uv",
"args": [
"run",
"/path/to/omni-video-mcp/server.py"
],
"env": {
"ELEVENLABS_API_KEY": "your_api_key_here"
}
}
}
}Note: The ELEVENLABS_API_KEY is currently required for high-fidelity word-level transcription mapping during ingestion.
When the agent uses this MCP server, it follows a 4-phase architecture:
Phase 1: Ingestion (omni_video_ingest)
The agent scans your raw .mp4 / .mov files, extracting a packed markdown transcript and an initial Visual Scene Graph.
Phase 2: Director's Cut (omni_video_preview)
The agent uses the transcript to construct an EDL (Edit Decision List) of the best takes. Ambiguous cuts can be visually verified by generating filmstrip PNGs via the preview tool.
Phase 3: VFX (omni_video_generate_vfx)
The agent generates HTML/CSS motion graphics (lower thirds, b-roll layouts) and the server renders them deterministically into transparent .webm videos via Hyperframes.
Phase 4: Sweetening & Render (omni_video_render)
The agent passes the EDL, VFX timestamps, and render settings to the server, which builds a complex FFmpeg graph to concatenate the footage, grade it, restore the audio, and export a final master.
Contributions are welcome! If you're adding new render pipeline capabilities (like auto-tracking or local whisper fallbacks), please open a PR. Ensure that any added Python dependencies are added to the pyproject.toml using uv add <package>.
MIT License
Renders motion graphics (e.g., lower thirds, titles) using Hyperframes. Returns the path to the rendered transparent .mov or .webm file.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry full weight. It discloses the use of Hyperframes and output format, but lacks details on side effects, authorization needs, or error behaviors. Adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action and output. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is relatively simple and has an output schema. The description covers purpose and basic behavior but omits when to use, parameter details, and potential prerequisites. Adequate but with gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has one parameter (request) with a nested object that is described in the schema's $defs. However, the tool description does not mention parameters or add meaning beyond the schema. With 0% schema description coverage for the top-level parameter, the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Renders motion graphics) and resource (e.g., lower thirds, titles) and specifies the output (path to .mov or .webm). It effectively distinguishes from sibling tools like omni_video_ingest, preview, and render.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for motion graphics generation but does not provide explicit guidance on when to use this tool versus siblings, nor does it list prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Ingests a directory of video files, generates word-level audio transcripts, and constructs a semantic Visual Scene Graph for B-Roll searching. Returns the path to the generated project metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It states that the tool ingests video, generates transcripts and scene graphs, and returns a metadata path. However, it does not mention whether files are modified, destructive potential, authentication needs, or time/resource consumption. The description is adequate but lacks depth for a full behavioral profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the tool's purpose, operations, and output. Every phrase adds value: ingestion, transcript generation, scene graph construction, B-roll searching goal, and return of metadata path. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main steps and purpose, referencing the output (metadata path) which is further defined by an output schema. It is clear relative to sibling tools. Minor gaps include lack of supported video formats or directory considerations, but overall sufficient for understanding core functionality.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter, directory_path, with a description in the schema itself. The tool description adds that the directory contains 'raw video footage' and that ingestion produces transcript and scene graph, but it does not provide additional semantics beyond what the schema already states. Baseline score of 3 is appropriate given high schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs ('Ingests', 'generates', 'constructs') and resources ('directory of video files', 'audio transcripts', 'Visual Scene Graph') to clearly state the tool's function. It also distinguishes from siblings (generate_vfx, preview, render) by naming the distinct outputs for B-Roll searching.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used for ingesting video to support B-roll searching, but it does not explicitly state when to use or avoid it versus alternatives like omni_video_generate_vfx or omni_video_preview. No prerequisites or excluded cases are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generates a filmstrip PNG of the specified video segment. Useful for visually verifying cut boundaries or B-roll placement. Returns the absolute path to the generated PNG file.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It discloses return value (absolute path to PNG) but omits side effects, error conditions, performance implications, or what happens with invalid inputs. Minimal behavioral disclosure beyond output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, each adding unique value: generation action, use case, return information. No wasted words, front-loaded with core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present (implied) and comprehensive parameter descriptions, the tool is mostly complete. Could mention error handling or performance constraints for large videos. Still strong given context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides descriptions for all parameters (file_path, start_time, end_time) achieving 100% coverage. The description adds no additional parameter semantics beyond noting the return type. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'generates' and resource 'filmstrip PNG of the specified video segment'. Intent is further clarified with use case for verifying cut boundaries or B-roll placement. Distinguishes from sibling tools (generate_vfx, ingest, render) by focusing on preview.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides explicit use cases: 'visually verifying cut boundaries or B-roll placement'. Does not explicitly state when not to use or mention alternatives, but context with sibling tools makes the purpose clear. Slightly improved by adding 'when not to use'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Orchestrates the final render pipeline: applies EDL cuts, overlay graphics, LUT color grading, audio restoration, and subtitle burning. Returns the path to the final rendered video.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It lists applied operations and notes return of a path, but does not disclose side effects like file writing at output_path, auth requirements, or potential destructive behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence efficiently lists the tool's actions and return value. It is front-loaded with the verb 'orchestrates'. Could be slightly improved by separating the list for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity, the description covers the high-level pipeline and key parameters. It lacks details on EDL JSON structure or output schema, but the separate schema and output schema exist to fill gaps, making it nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description maps 'EDL cuts' to edl_json, 'LUT color grading' to lut_path, and 'subtitle burning' to subtitle_style, adding context beyond the schema. However, it omits output_path and mentions overlay graphics/audio restoration not directly reflected in parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool orchestrates the final render pipeline, listing specific operations (EDL cuts, overlay graphics, LUT grading, audio restoration, subtitle burning) and returns the output path. It distinctly separates from sibling tools like omni_video_preview by focusing on the final render stage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage as the final render step after other pipeline stages, but lacks explicit when-to-use or when-not-to-use guidance, particularly differentiating from omni_video_preview which may serve as a preview step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Each tool serves a distinct function: ingest creates metadata, generate_vfx produces graphics, preview provides visual verification, and render finalizes output. No overlapping purposes.
All tools follow the 'omni_video_' prefix with a verb describing the action (ingest, generate_vfx, preview, render), forming a clear and predictable pattern.
Four tools cover the essential phases of video production—ingestion, effect creation, previewing, and rendering—without being excessive or insufficient.
The tool set covers the core pipeline, but lacks explicit tools for timeline editing or asset selection, requiring reliance on the render tool for EDL processing.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
AI video editor: create projects, edit timelines, add captions and effects, and render videos.
Edit video by talking to your AI — search footage, cut timelines, apply effects, add captions.
A real timeline video editor for AI agents: journaled edits, FFmpeg/MLT rendering, exports
Author, run, and publish AI video pipelines. Agents can pay per render with x402 USDC, no API key.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Mallet-Builds/omni-video-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
