Skip to main content
Glama

VideoGenerationMCP

A FastMCP server that gives agents a validated interface to Kling Omni + Seedance 2 video generation (via PiAPI) and ElevenLabs voiceover — with every provider constraint enforced locally (Pydantic) before any paid API call.

Tools

Tool

Purpose

generate_kling_video

Kling Omni single- or multi-shot generation

generate_seedance_video

Seedance 2 generation; auto-chains the Hebrew BVAC lipsync pipeline

generate_seedance_first_last

Seedance first/last-frame interpolation

verify_generated_audio

Scribe QA gate on a finished task's video (async jobs)

generate_elevenlabs_voiceover

ElevenLabs TTS with character-level timestamps

transliterate_hebrew

Hebrew → Latin (LLM-backed) for lipsync prompts

list_voices

List ElevenLabs voices

get_task

Poll any PiAPI task

upload_asset · list_assets · get_asset · delete_asset

PiAPI private asset library (reusable asset:// persona refs)

split_audio

Cut a master voiceover at timestamps → per-clip segments

extract_frame

Grab a frame (default: last) for first/last-frame clip bridging

stitch_videos

Concat clips (normalized) into the final multi-clip ad

trim_video

Frame-accurate cut of a clip to an exact span (re-encode)

retime_video

Stretch/compress a clip to a target duration (PTS, optional interpolation)

mix_narration

Lay a voiceover as primary audio over (silent) video, optional ducked bed

host_file

Host a local file on a temporary public URL (non-persona refs)

burn_captions

Word-timed captions via Scribe (RTL-correct Hebrew) burned onto a video

generate_music

Eleven Music instrumental bed from a text prompt (3–600s)

generate_sound_effect

ElevenLabs Text-to-Sound-Effects ambient/diegetic bed (0.5–30s)

mix_music_into_video

Lay a music bed under speech: low gain + side-chain ducking

Highlights: Hebrew BVAC lipsync (ElevenLabs eleven_v3 → ffmpeg black-video carrier → Seedance omni_reference) with two Scribe audio gates; @-tag reference validation; a content gate (blocks minor/real-person prompts); private-asset support on the less-restriction tier; 720p default (1080p on request).

Related MCP server: Aetherwave Studio

Requirements

  • Python ≥ 3.12 and uv

  • ffmpeg on PATH (brew install ffmpeg / apt install ffmpeg)

  • PIAPI_KEY and ELEVENLABS_KEY (see .env.example)

  • Optional: OPENROUTER_API_KEY (fallback for Hebrew transliteration; primary is a local LMStudio model, default google/gemma-4-e4b)

Connect to Claude Code

No clone needed — uvx installs and runs the server straight from this repo:

claude mcp add video \
  --env PIAPI_KEY=your_piapi_key \
  --env ELEVENLABS_KEY=your_elevenlabs_key \
  --env OPENROUTER_API_KEY=your_openrouter_key \
  -- uvx --from git+https://github.com/AvivK5498/VideoGenerationMCP video-mcp

Or add it to a project .mcp.json:

{
  "mcpServers": {
    "video": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/AvivK5498/VideoGenerationMCP", "video-mcp"],
      "env": { "PIAPI_KEY": "…", "ELEVENLABS_KEY": "…", "OPENROUTER_API_KEY": "…" }
    }
  }
}

Verify with /mcp inside Claude Code.

Connect to Codex

Add to ~/.codex/config.toml:

[mcp_servers.video]
command = "uvx"
args = ["--from", "git+https://github.com/AvivK5498/VideoGenerationMCP", "video-mcp"]
env = { PIAPI_KEY = "…", ELEVENLABS_KEY = "…", OPENROUTER_API_KEY = "…" }

(or codex mcp add video -- uvx --from git+https://github.com/AvivK5498/VideoGenerationMCP video-mcp). Codex MCP servers communicate over stdio; restart Codex to pick up the config.

Local development

git clone git@github.com:AvivK5498/VideoGenerationMCP.git
cd VideoGenerationMCP
uv sync
cp .env.example .env   # fill in PIAPI_KEY + ELEVENLABS_KEY
uv run pytest -q       # 203 tests

Run standalone (stdio): uv run video-mcp. To wire a local checkout instead of the git install, swap the command for uv run --directory /ABSOLUTE/PATH/TO/VideoGenerationMCP video-mcp.

More

  • CONTRACT.md — full interface spec for every module.

  • samples/payloads.md — ready-to-use tool-call examples.

  • scripts/ — live end-to-end drivers used to validate against PiAPI/ElevenLabs.

Available Tools

24 tools
burn_captionsA

Burn word-timed captions onto a video (local path, URL, or PiAPI task_id).

Default flow transcribes the video's OWN audio with ElevenLabs Scribe (word timestamps), groups words into short chunks (max_words per caption), and burns styled overlays: white bold, black stroke, drop shadow, centered low-third, Hebrew rendered in correct RTL visual order (English/brand tokens stay LTR). Pass captions ([{text, start, end}]) to skip Scribe and burn exact chunks; style overrides (font_px, y_pct, fill, crf, font_path...). Run AFTER verify_generated_audio passes and BEFORE stitch_videos.

Inputs: style, video, task_id, captions, +3 more. Returns: object

delete_assetB

Delete a private asset.

Inputs: asset_id (required). Returns: object

detect_beatsA

Detect tempo and beat times of an audio track — pure analysis, no output.

Returns {"bpm": float, "beats": [seconds, ...]} with beat times ascending from the track start and covering the whole file. The tempo is reported at its lower octave, pinned into [min_bpm, max_bpm] (a track that reads as 172 BPM comes back as ~86) — pass the window to control which octave the grid lands on. Use this to beat-snap multi-clip seams onto a music bed before stitching.

Silence / no detectable pulse returns {"bpm": 0.0, "beats": []} (not an error).

Inputs: audio_path (required), max_bpm, min_bpm. Returns: object

extract_frameA

Save one frame of a video (local path or URL) as PNG; default = LAST frame.

Use the last frame of clip N as image_first of clip N+1 (generate_seedance_first_last) to bridge multi-clip continuity. With upload=true the frame is also hosted on a temporary public URL so it can be passed as a reference directly.

Inputs: video (required), time_s, upload. Returns: object

generate_elevenlabs_voiceoverB

Synthesize speech with ElevenLabs and write the audio to a temp file.

Inputs: text (required), voice_id (required), seed, speed, +11 more. Returns: object

generate_kling_videoC

Generate a Kling Omni video (single prompt or multi-shot).

Inputs: wait, shots, video, images, +9 more. Returns: object

generate_musicA

Compose a music track with Eleven Music (3-600s) from a text prompt.

For speech ads the bed must be COMPLEMENTARY, not the main event: prompt for minimal/low-key/background music ("soft, sparse, no melodic hook, low intensity"), keep force_instrumental=true so it never fights the voiceover, match duration_s to the video, then lay it under with mix_music_into_video (low gain + speech ducking).

Inputs: prompt (required), duration_s (required), model_id, output_path, +1 more. Returns: object

generate_seedance_first_lastB

Generate a Seedance video from a first (and optional last) frame image.

Inputs: prompt (required), image_first (required), wait, duration, +4 more. Returns: object

generate_seedance_videoA

Generate a Seedance video; auto-chain Hebrew BVAC lipsync when language is Hebrew.

HEBREW (language="he") — this tool runs the ENTIRE BVAC chain itself; do NOT pre-build anything:

  • Speech: synthesized from text (keep it Hebrew) with ElevenLabs eleven_v3 and voice_id — OR pass audio_path (local mp3) to use a pre-approved take and skip TTS. Audition flow: generate_elevenlabs_voiceover -> user approves -> pass its audio_path here. Never build a black carrier, upload audio, or register a carrier asset yourself; the tool does all of it and ignores yours.

  • Prompt: pass ONLY the scene description, Latin-only (transliterate_hebrew first). The "@ImageN is ..." / "@Video1 ..." reference lines and the lip-sync mechanism are composed server-side — do not write them yourself (customize via image_roles / other_roles).

  • romanized_text: ALWAYS supply your own PHONEMIC RESPELLING of text — spell the Hebrew the way an English reader sounds it out, because Seedance picks visemes from this text with an English-dominant classifier (linguistic romanization makes it mouth English: chazir -> "church"). Form: syllable- hyphenated, stressed syllable in CAPS, ' for schwa — חזיר -> khah-ZEER (NOT chazir); גבר, קום מהספה -> GEH-ver, koom meh-hah-SAH-pah; כושר -> KOH-sher. Every syllable sayable; gender/morphology respected but rendered by English sound; English/brand tokens byte-for-byte; sentence-final words with extra care (they drive the lip-sync hardest). A structural gate validates it; omit it and an LLM (OpenRouter-first) respells instead.

  • task_type is forced to seedance-2-less-restriction: the lower-moderation tier required for asset-backed fictional personas. Requesting another type has no effect; this is intentional, not an error.

  • Scribe QA gates run on the source audio, and on the generated video when wait=true.

References split into human_image_urls (faces/people) and other_image_urls (product/room/scene). Generated people MUST be private assets: human refs have to be asset://<id> (register them with upload_asset first) unless auto_upload_assets=true. image_urls is a legacy alias for human_image_urls.

Inputs: prompt (required), mode, text, wait, +20 more. Returns: object

generate_sound_effectA

Generate an ambient/diegetic sound-effect bed (0.5-30s) from a text prompt.

Generation only — the caller mixes the bed onto the video locally with its own ducking recipe. Describe the soundscape concretely ("busy gym ambience: muffled crowd murmur, low machine hum"), pass the clip/ad runtime as duration_seconds, and keep loop=true for a seamless bed.

Inputs: prompt (required), duration_seconds (required), loop, output_path, +1 more. Returns: object

get_assetA

Get a single asset's current state (refreshes its TTL).

Inputs: asset_id (required). Returns: object

get_taskB

Fetch the current state of a PiAPI task.

Inputs: task_id (required). Returns: object

host_fileA

Host a local file on a temporary public URL (tmpfiles, ~1h retention).

Use for non-asset references that need a provider-fetchable URL: a local ElevenLabs mp3 going into audio_urls (English reference-audio lip-sync), a product/room photo for other_image_urls, etc. Human/persona refs do NOT go here — register those with upload_asset instead.

Inputs: path (required). Returns: object

list_assetsB

List private assets (optionally filter by status: active,processing,failed).

Inputs: page, size, status. Returns: object

list_voicesA

List available ElevenLabs voices.

Inputs: no named parameters. Returns: object

mix_music_into_videoA

Lay a music bed under a video's existing speech/ambience audio.

Gain is ADAPTIVE by default: both tracks are loudness-measured and the music sits music_below_speech_db LUFS below the speech (14 = clearly audible but secondary; 18-20 = barely-there). duck=true adds a gentle side-chain dip while someone talks. Pass music_gain_db only to force a fixed gain. Music for speech ads is complementary, not the main event. Video stream is copied untouched; inputs may be URLs.

Inputs: music (required), video (required), output_path (required), duck, +2 more. Returns: object

mix_narrationA

Lay a voiceover as the PRIMARY audio over a (silent) video.

The VO plays at full level and the video stream is copied untouched. An optional bed_path (ambient/music) is mixed bed_below_voice_db LUFS under the VO with a gentle side-chain duck. This is the inverse of mix_music_into_video (which ducks a bed under speech ALREADY in the video). Output runs the video's length; the audio is padded with silence if shorter, trimmed if longer. ffprobe-verified.

Inputs: video_path (required), output_path (required), voiceover_path (required), bed_path, +1 more. Returns: object

retime_videoA

Stretch/compress a clip to hit a target duration (or explicit speed).

Provide EITHER target_duration_s (speed is computed as source/target) OR speed (1.0 unchanged, 0.5 = half-speed/2x longer). Use this to slow a b-roll clip slightly to fill a voiceover segment without repeating footage. interpolate=true smooths slow-mo via motion interpolation (default off = frame duplication). speed is clamped to [0.5, 2.0]; outside that range is a ToolError. Audio (if any) is retimed too. ffprobe-verified.

Inputs: video_path (required), output_path (required), speed, interpolate, +1 more. Returns: object

split_audioA

Cut a master voiceover mp3 at split_points_s (seconds, strictly increasing).

Returns len(points)+1 ordered segments with paths and durations. Workflow for ads longer than one clip: generate ONE master VO with generate_elevenlabs_voiceover (with_timestamps=true), pick sentence-boundary timestamps, split here, then pass each segment's path as audio_path to its own generate_seedance_video call. Each segment must fit its clip duration.

Inputs: audio_path (required), split_points_s (required), output_dir. Returns: object

stitch_videosA

Concatenate >= 2 clips (local paths or URLs, in order) into one MP4.

Hard cuts; every clip is normalized to the first clip's resolution/fps so mixed specs concat cleanly. Use this for the final assembly of a multi-clip ad — do not shell out to ffmpeg concat yourself.

Inputs: videos (required), output_path (required). Returns: object

transliterate_hebrewA

Transliterate Hebrew text to Latin (via LMStudio/OpenRouter LLM) for visual prompts.

Inputs: text (required). Returns: object

trim_videoA

Frame-accurately cut a clip to an exact span (re-encode, not -c copy).

Provide EITHER duration_s (keep [0, duration_s]) OR start_s+end_s. Use this to cut a generated b-roll clip down to a voiceover segment's length. Resolution/fps/aspect are preserved; a silent clip stays silent. Returns the ffprobe-verified actual duration.

Inputs: video_path (required), output_path (required), end_s, start_s, +1 more. Returns: object

upload_assetA

Register a private asset from a local path or public URL; return its asset://id.

Local paths are first uploaded to a temporary public host for ingestion (the asset persists on PiAPI ~days afterward, independent of that URL). If influencer_page (path to an Obsidian .md) is given, the asset id is recorded there under a managed section for reuse.

Inputs: image (required), name, asset_type, wait_active, +1 more. Returns: object

verify_generated_audioA

Run the generated-video Scribe QA gate on a finished Seedance task.

Use this after an async (wait=false) Hebrew lipsync job completes: pass the original spoken text (Hebrew) plus the task_id (or a direct video_url). Downloads the video, extracts its audio, transcribes with ElevenLabs Scribe, and judges against text (100% pass / 85-99% warning / <85% raises). Do not substitute local whisper or other ASR — this is the canonical gate.

Inputs: text (required), task_id, video_url. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.5/5.0

    Scored across 24 tools

    Disambiguation4/5

    Most tools have clear, distinct purposes. The two Seedance generation tools (generate_seedance_video and generate_seedance_first_last) and the two audio-mixing tools (mix_music_into_video and mix_narration) have some overlap, but their descriptions clarify the different use cases.

    Naming Consistency5/5

    All tool names follow a consistent verb_noun snake_case pattern (e.g., list_assets, generate_kling_video, upload_asset). There are no mixed conventions or inconsistent verb styles.

    Tool Count4/5

    At 24 tools, the count is slightly above the typical 3-15 range but justifiable given the server's comprehensive video-generation and post-production scope. Each tool serves a distinct function in the pipeline, so none feel redundant.

    Completeness4/5

    The server covers the full video creation workflow: generation, voiceover, music, SFX, asset management, editing, mixing, and captions. Minor gaps exist, such as no list_tasks or cancel_task, but these do not significantly impede the core use case.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      One MCP server for music, image, video, and audio generation across Suno, Grok Imagine, Seedance, Kling, Hailuo, Wan, VEO, Ideogram, and GPT Image 2. Generate, edit, upscale, reframe, and master through one API key and one credit pool.
      16
      1,274 npm
      6
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      MCP server for Pixmax API enabling generation of images, video, text, audio, and 3D across dozens of models like Midjourney, Kling, and ElevenLabs.
      10
      MIT