VideoGenerationMCP
Provides tools for text-to-speech voiceover generation via ElevenLabs, including character-level timestamps and voice listing capabilities.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@VideoGenerationMCP create a Kling Omni video of a cat playing with yarn"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
VideoGenerationMCP
A FastMCP server that gives agents a validated interface to Kling Omni + Seedance 2 video generation (via PiAPI) and ElevenLabs voiceover — with every provider constraint enforced locally (Pydantic) before any paid API call.
Tools
Tool | Purpose |
| Kling Omni single- or multi-shot generation |
| Seedance 2 generation; auto-chains the Hebrew BVAC lipsync pipeline |
| Seedance first/last-frame interpolation |
| Scribe QA gate on a finished task's video (async jobs) |
| ElevenLabs TTS with character-level timestamps |
| Hebrew → Latin (LLM-backed) for lipsync prompts |
| List ElevenLabs voices |
| Poll any PiAPI task |
| PiAPI private asset library (reusable |
| Cut a master voiceover at timestamps → per-clip segments |
| Grab a frame (default: last) for first/last-frame clip bridging |
| Concat clips (normalized) into the final multi-clip ad |
| Frame-accurate cut of a clip to an exact span (re-encode) |
| Stretch/compress a clip to a target duration (PTS, optional interpolation) |
| Lay a voiceover as primary audio over (silent) video, optional ducked bed |
| Host a local file on a temporary public URL (non-persona refs) |
| Word-timed captions via Scribe (RTL-correct Hebrew) burned onto a video |
| Eleven Music instrumental bed from a text prompt (3–600s) |
| ElevenLabs Text-to-Sound-Effects ambient/diegetic bed (0.5–30s) |
| Lay a music bed under speech: low gain + side-chain ducking |
Highlights: Hebrew BVAC lipsync (ElevenLabs eleven_v3 → ffmpeg black-video carrier →
Seedance omni_reference) with two Scribe audio gates; @-tag reference validation;
a content gate (blocks minor/real-person prompts); private-asset support on the
less-restriction tier; 720p default (1080p on request).
Related MCP server: Aetherwave Studio
Requirements
Python ≥ 3.12 and
uvffmpegon PATH (brew install ffmpeg/apt install ffmpeg)PIAPI_KEYandELEVENLABS_KEY(see.env.example)Optional:
OPENROUTER_API_KEY(fallback for Hebrew transliteration; primary is a local LMStudio model, defaultgoogle/gemma-4-e4b)
Connect to Claude Code
No clone needed — uvx installs and runs the server straight from this repo:
claude mcp add video \
--env PIAPI_KEY=your_piapi_key \
--env ELEVENLABS_KEY=your_elevenlabs_key \
--env OPENROUTER_API_KEY=your_openrouter_key \
-- uvx --from git+https://github.com/AvivK5498/VideoGenerationMCP video-mcpOr add it to a project .mcp.json:
{
"mcpServers": {
"video": {
"command": "uvx",
"args": ["--from", "git+https://github.com/AvivK5498/VideoGenerationMCP", "video-mcp"],
"env": { "PIAPI_KEY": "…", "ELEVENLABS_KEY": "…", "OPENROUTER_API_KEY": "…" }
}
}
}Verify with /mcp inside Claude Code.
Connect to Codex
Add to ~/.codex/config.toml:
[mcp_servers.video]
command = "uvx"
args = ["--from", "git+https://github.com/AvivK5498/VideoGenerationMCP", "video-mcp"]
env = { PIAPI_KEY = "…", ELEVENLABS_KEY = "…", OPENROUTER_API_KEY = "…" }(or codex mcp add video -- uvx --from git+https://github.com/AvivK5498/VideoGenerationMCP video-mcp).
Codex MCP servers communicate over stdio; restart Codex to pick up the config.
Local development
git clone git@github.com:AvivK5498/VideoGenerationMCP.git
cd VideoGenerationMCP
uv sync
cp .env.example .env # fill in PIAPI_KEY + ELEVENLABS_KEY
uv run pytest -q # 203 testsRun standalone (stdio): uv run video-mcp. To wire a local checkout instead of the
git install, swap the command for
uv run --directory /ABSOLUTE/PATH/TO/VideoGenerationMCP video-mcp.
More
CONTRACT.md— full interface spec for every module.samples/payloads.md— ready-to-use tool-call examples.scripts/— live end-to-end drivers used to validate against PiAPI/ElevenLabs.
Available Tools
24 toolsburn_captionsA
Burn word-timed captions onto a video (local path, URL, or PiAPI task_id).
Default flow transcribes the video's OWN audio with ElevenLabs Scribe (word
timestamps), groups words into short chunks (max_words per caption), and
burns styled overlays: white bold, black stroke, drop shadow, centered
low-third, Hebrew rendered in correct RTL visual order (English/brand
tokens stay LTR). Pass captions ([{text, start, end}]) to skip Scribe and
burn exact chunks; style overrides (font_px, y_pct, fill, crf, font_path...).
Run AFTER verify_generated_audio passes and BEFORE stitch_videos.
Inputs: style, video, task_id, captions, +3 more. Returns: object
delete_assetB
Delete a private asset.
Inputs: asset_id (required). Returns: object
detect_beatsA
Detect tempo and beat times of an audio track — pure analysis, no output.
Returns {"bpm": float, "beats": [seconds, ...]} with beat times ascending from the track start and covering the whole file. The tempo is reported at its lower octave, pinned into [min_bpm, max_bpm] (a track that reads as 172 BPM comes back as ~86) — pass the window to control which octave the grid lands on. Use this to beat-snap multi-clip seams onto a music bed before stitching.
Silence / no detectable pulse returns {"bpm": 0.0, "beats": []} (not an error).
Inputs: audio_path (required), max_bpm, min_bpm. Returns: object
extract_frameA
Save one frame of a video (local path or URL) as PNG; default = LAST frame.
Use the last frame of clip N as image_first of clip N+1
(generate_seedance_first_last) to bridge multi-clip continuity. With
upload=true the frame is also hosted on a temporary public URL so it can be
passed as a reference directly.
Inputs: video (required), time_s, upload. Returns: object
generate_elevenlabs_voiceoverB
Synthesize speech with ElevenLabs and write the audio to a temp file.
Inputs: text (required), voice_id (required), seed, speed, +11 more. Returns: object
generate_kling_videoC
Generate a Kling Omni video (single prompt or multi-shot).
Inputs: wait, shots, video, images, +9 more. Returns: object
generate_musicA
Compose a music track with Eleven Music (3-600s) from a text prompt.
For speech ads the bed must be COMPLEMENTARY, not the main event: prompt for minimal/low-key/background music ("soft, sparse, no melodic hook, low intensity"), keep force_instrumental=true so it never fights the voiceover, match duration_s to the video, then lay it under with mix_music_into_video (low gain + speech ducking).
Inputs: prompt (required), duration_s (required), model_id, output_path, +1 more. Returns: object
generate_seedance_first_lastB
Generate a Seedance video from a first (and optional last) frame image.
Inputs: prompt (required), image_first (required), wait, duration, +4 more. Returns: object
generate_seedance_videoA
Generate a Seedance video; auto-chain Hebrew BVAC lipsync when language is Hebrew.
HEBREW (language="he") — this tool runs the ENTIRE BVAC chain itself; do NOT pre-build anything:
Speech: synthesized from
text(keep it Hebrew) with ElevenLabs eleven_v3 andvoice_id— OR passaudio_path(local mp3) to use a pre-approved take and skip TTS. Audition flow: generate_elevenlabs_voiceover -> user approves -> pass its audio_path here. Never build a black carrier, upload audio, or register a carrier asset yourself; the tool does all of it and ignores yours.Prompt: pass ONLY the scene description, Latin-only (transliterate_hebrew first). The "@ImageN is ..." / "@Video1 ..." reference lines and the lip-sync mechanism are composed server-side — do not write them yourself (customize via image_roles / other_roles).
romanized_text: ALWAYS supply your own PHONEMIC RESPELLING oftext— spell the Hebrew the way an English reader sounds it out, because Seedance picks visemes from this text with an English-dominant classifier (linguistic romanization makes it mouth English: chazir -> "church"). Form: syllable- hyphenated, stressed syllable in CAPS, ' for schwa — חזיר -> khah-ZEER (NOT chazir); גבר, קום מהספה -> GEH-ver, koom meh-hah-SAH-pah; כושר -> KOH-sher. Every syllable sayable; gender/morphology respected but rendered by English sound; English/brand tokens byte-for-byte; sentence-final words with extra care (they drive the lip-sync hardest). A structural gate validates it; omit it and an LLM (OpenRouter-first) respells instead.task_type is forced to seedance-2-less-restriction: the lower-moderation tier required for asset-backed fictional personas. Requesting another type has no effect; this is intentional, not an error.
Scribe QA gates run on the source audio, and on the generated video when wait=true.
References split into human_image_urls (faces/people) and other_image_urls
(product/room/scene). Generated people MUST be private assets: human refs have
to be asset://<id> (register them with upload_asset first) unless
auto_upload_assets=true. image_urls is a legacy alias for human_image_urls.
Inputs: prompt (required), mode, text, wait, +20 more. Returns: object
generate_sound_effectA
Generate an ambient/diegetic sound-effect bed (0.5-30s) from a text prompt.
Generation only — the caller mixes the bed onto the video locally with its own ducking recipe. Describe the soundscape concretely ("busy gym ambience: muffled crowd murmur, low machine hum"), pass the clip/ad runtime as duration_seconds, and keep loop=true for a seamless bed.
Inputs: prompt (required), duration_seconds (required), loop, output_path, +1 more. Returns: object
get_assetA
Get a single asset's current state (refreshes its TTL).
Inputs: asset_id (required). Returns: object
get_taskB
Fetch the current state of a PiAPI task.
Inputs: task_id (required). Returns: object
host_fileA
Host a local file on a temporary public URL (tmpfiles, ~1h retention).
Use for non-asset references that need a provider-fetchable URL: a local
ElevenLabs mp3 going into audio_urls (English reference-audio lip-sync),
a product/room photo for other_image_urls, etc. Human/persona refs do NOT
go here — register those with upload_asset instead.
Inputs: path (required). Returns: object
list_assetsB
List private assets (optionally filter by status: active,processing,failed).
Inputs: page, size, status. Returns: object
list_voicesA
List available ElevenLabs voices.
Inputs: no named parameters. Returns: object
mix_music_into_videoA
Lay a music bed under a video's existing speech/ambience audio.
Gain is ADAPTIVE by default: both tracks are loudness-measured and the
music sits music_below_speech_db LUFS below the speech (14 = clearly
audible but secondary; 18-20 = barely-there). duck=true adds a gentle
side-chain dip while someone talks. Pass music_gain_db only to force a
fixed gain. Music for speech ads is complementary, not the main event.
Video stream is copied untouched; inputs may be URLs.
Inputs: music (required), video (required), output_path (required), duck, +2 more. Returns: object
mix_narrationA
Lay a voiceover as the PRIMARY audio over a (silent) video.
The VO plays at full level and the video stream is copied untouched. An
optional bed_path (ambient/music) is mixed bed_below_voice_db LUFS
under the VO with a gentle side-chain duck. This is the inverse of
mix_music_into_video (which ducks a bed under speech ALREADY in the
video). Output runs the video's length; the audio is padded with silence
if shorter, trimmed if longer. ffprobe-verified.
Inputs: video_path (required), output_path (required), voiceover_path (required), bed_path, +1 more. Returns: object
retime_videoA
Stretch/compress a clip to hit a target duration (or explicit speed).
Provide EITHER target_duration_s (speed is computed as source/target)
OR speed (1.0 unchanged, 0.5 = half-speed/2x longer). Use this to slow
a b-roll clip slightly to fill a voiceover segment without repeating
footage. interpolate=true smooths slow-mo via motion interpolation
(default off = frame duplication). speed is clamped to [0.5, 2.0]; outside
that range is a ToolError. Audio (if any) is retimed too. ffprobe-verified.
Inputs: video_path (required), output_path (required), speed, interpolate, +1 more. Returns: object
split_audioA
Cut a master voiceover mp3 at split_points_s (seconds, strictly increasing).
Returns len(points)+1 ordered segments with paths and durations. Workflow for
ads longer than one clip: generate ONE master VO with
generate_elevenlabs_voiceover (with_timestamps=true), pick sentence-boundary
timestamps, split here, then pass each segment's path as audio_path to its
own generate_seedance_video call. Each segment must fit its clip duration.
Inputs: audio_path (required), split_points_s (required), output_dir. Returns: object
stitch_videosA
Concatenate >= 2 clips (local paths or URLs, in order) into one MP4.
Hard cuts; every clip is normalized to the first clip's resolution/fps so mixed specs concat cleanly. Use this for the final assembly of a multi-clip ad — do not shell out to ffmpeg concat yourself.
Inputs: videos (required), output_path (required). Returns: object
transliterate_hebrewA
Transliterate Hebrew text to Latin (via LMStudio/OpenRouter LLM) for visual prompts.
Inputs: text (required). Returns: object
trim_videoA
Frame-accurately cut a clip to an exact span (re-encode, not -c copy).
Provide EITHER duration_s (keep [0, duration_s]) OR start_s+end_s.
Use this to cut a generated b-roll clip down to a voiceover segment's
length. Resolution/fps/aspect are preserved; a silent clip stays silent.
Returns the ffprobe-verified actual duration.
Inputs: video_path (required), output_path (required), end_s, start_s, +1 more. Returns: object
upload_assetA
Register a private asset from a local path or public URL; return its asset://id.
Local paths are first uploaded to a temporary public host for ingestion (the
asset persists on PiAPI ~days afterward, independent of that URL). If
influencer_page (path to an Obsidian .md) is given, the asset id is recorded
there under a managed section for reuse.
Inputs: image (required), name, asset_type, wait_active, +1 more. Returns: object
verify_generated_audioA
Run the generated-video Scribe QA gate on a finished Seedance task.
Use this after an async (wait=false) Hebrew lipsync job completes: pass the
original spoken text (Hebrew) plus the task_id (or a direct video_url).
Downloads the video, extracts its audio, transcribes with ElevenLabs Scribe,
and judges against text (100% pass / 85-99% warning / <85% raises).
Do not substitute local whisper or other ASR — this is the canonical gate.
Inputs: text (required), task_id, video_url. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 24 tools
Most tools have clear, distinct purposes. The two Seedance generation tools (generate_seedance_video and generate_seedance_first_last) and the two audio-mixing tools (mix_music_into_video and mix_narration) have some overlap, but their descriptions clarify the different use cases.
All tool names follow a consistent verb_noun snake_case pattern (e.g., list_assets, generate_kling_video, upload_asset). There are no mixed conventions or inconsistent verb styles.
At 24 tools, the count is slightly above the typical 3-15 range but justifiable given the server's comprehensive video-generation and post-production scope. Each tool serves a distinct function in the pipeline, so none feel redundant.
The server covers the full video creation workflow: generation, voiceover, music, SFX, asset management, editing, mixing, and captions. Minor gaps exist, such as no list_tasks or cancel_task, but these do not significantly impede the core use case.
Maintenance
Related MCP Connectors
MCP server for Kling AI video generation
MCP server for Google Veo AI video generation
MCP server for Wan AI video generation
MCP server for Hailuo (MiniMax) AI video generation
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceMCP server for 4K video generation using Google VEO 3.1 — text-to-video, image-to-video, video extension, and frame interpolation.3MIT
- AlicenseAqualityBmaintenanceOne MCP server for music, image, video, and audio generation across Suno, Grok Imagine, Seedance, Kling, Hailuo, Wan, VEO, Ideogram, and GPT Image 2. Generate, edit, upscale, reframe, and master through one API key and one credit pool.161,274 npm6MIT
- AlicenseAqualityCmaintenanceMCP server for generating, editing, and batch processing videos using xAI's Grok Imagine Video API, with support for text-to-video, image-to-video, and video editing via natural language prompts.4116 npm1MIT
- AlicenseAqualityBmaintenanceMCP server for Pixmax API enabling generation of images, video, text, audio, and 3D across dozens of models like Midjourney, Kling, and ElevenLabs.10MIT