Skip to main content
Glama

Audio To Video

audio_to_video

Generate video from audio and text prompt.

Creates videos conditioned on audio content. The audio drives the video generation, allowing synchronized audio-visual output. Optionally accepts anchor frame images.

IMPORTANT: Check model specifications using get_available_models() before calling. Pay attention to features.supports_guidance, features.supports_steps, and model limits.


Available models:

  • Ltx2_5_22B_Dist_INT8: size=512-1344x512-1344, fps=24 (fixed), frames=49-241, w/h rounded down to a multiple of 64, ≤1.03Mpx, side≤1344, aspect≤2, audio2video audio=1-10s, no negative_prompt

  • Ltx2_3_22B_Dist_INT8: size=512-1024x512-1024, fps=24 (fixed), frames=49-241, audio2video audio=1-11s, no negative_prompt

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "prompt": "<prompt>",
      "model": "<model>",
      "audio": "<audio>"
    },
    "name": "audio_to_video"
  }
}

Input Schema

NameRequiredDescriptionDefault
fpsNoFrames per second. Defaults to the model's own value (see the model list); most models fix it
seedNoRandom seed (-1 for random)
audioYesAudio file as URL, data URI (data:audio/mpeg;base64,...), or base64 string. Conditions the video generation on the audio content
modelYesVideo generation model name (must support audio2video inference type)
stepsNoInference steps. Only used if model's features.supports_steps is true. Limits per model (see the model list)
widthNoVideo width in pixels. Defaults to the model's own value (see the model list)
framesNoNumber of video frames. Defaults to the model's own value (see the model list); each model has its own range
heightNoVideo height in pixels. Defaults to the model's own value (see the model list)
promptYesText prompt for video generation
guidanceNoGuidance scale. Only used if model's features.supports_guidance is true. Limits per model (see the model list)
webhook_urlNoHTTPS URL for deAPI to call when this job finishes. Notifies your own systems; the tool still returns the result here
webhook_secretNo32-255 char key deAPI signs the callback with (HMAC-SHA256). Needs webhook_url. A secret given in chat stays in the transcript
negative_promptNoThings to exclude (optional)
last_frame_imageNoLast frame anchor image (URL, data URI, or base64, optional)
first_frame_imageNoFirst frame anchor image (URL, data URI, or base64, optional)

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does substantial work: it enumerates per-model hard limits (fps fixed at 24, frame ranges 49-241, audio length 1-10s/1-11s, ≤1.03Mpx, aspect≤2, dimensions rounded down to multiples of 64) and flags that one model rejects negative_prompt. It does not disclose whether the call blocks until completion or how long generation takes, and the presence of a sibling price tool means cost behavior is left implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, the IMPORTANT prerequisite is visually flagged, and the model specs sit in a scannable list. The per-model enumeration is long but earns its space by encoding hard constraints; minor redundancy between "Defaults to the model's own value (see the model list)" in the schema and the table header is acceptable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 15-parameter generative tool with an output schema and no annotations, the description supplies the prerequisite check and the per-model constraint matrix that no structured field conveys. Remaining gaps are execution semantics (sync vs. job/poll, typical latency) and cost signaling, which are relevant given the paired audio_to_video_price sibling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the model table adds real meaning the schema lacks: which parameters are silently ignored per model (negative_prompt unsupported, fps fixed, steps/guidance gated on feature flags) and concrete numeric ranges for frames, size, and audio duration. That constrains how an agent should set values rather than just restating field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first line gives a specific verb and resource ("Generate video from audio and text prompt") and immediately narrows scope with "Creates videos conditioned on audio content. The audio drives the video generation." That modality framing tells an agent how this differs from text_to_video or image_to_video siblings without needing their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit prerequisite and names the sibling to call first: "Check model specifications using get_available_models() before calling," plus which feature flags to inspect. What is missing is a direct statement of when to choose audio-conditioned generation over text_to_video/image_to_video, so the agent must infer the alternative's condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.