Skip to main content
Glama

Image To Video

image_to_video

Generate video from a static image or reference images, videos, and audio using text prompts.

Instructions

Generate video from static image(s), or from reference images, videos and audio.

Two modes, mutually exclusive:

  • Keyframes: first_frame_image (and optionally last_frame_image) is animated.

  • References: ref_images / ref_videos / ref_audios, on models that support them (the model list shows "refs: ..." for those). The prompt addresses each reference by its tag: <Picture 1>, <Picture 2> for images, <Video 1> for a video, <Audio 1> for audio, numbered from 1 within each type in the order given. Order matters: swapping two images swaps what the tags refer to. Example prompt: "<Picture 1> walks into the room from <Picture 2>, speaking with the voice of <Audio 1>".

Accepts files as:

  • Public URLs (recommended: chains with text_to_image, and the ONLY way in for a file of your own - chat clients do not pass files through to tools)

  • Data URIs or raw base64 (for callers that already hold the bytes)

IMPORTANT: Check model specifications using get_available_models() before calling. Pay attention to limits for width/height/frames/fps and, in reference mode, the max_ref_* counts and durations.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "prompt": "<prompt>",
      "model": "<model>"
    },
    "name": "image_to_video"
  }
}

Input Schema

NameRequiredDescriptionDefault
fpsNoFrames per second. Defaults to the model's own value (see the model list); most models fix it
seedNoRandom seed for reproducibility
modelYesVideo generation model name
stepsNoNumber of inference steps. Defaults to the model's own value (see the model list); ignored by models that do not support steps
widthNoVideo width in pixels. Defaults to the model's own value (see the model list)
framesNoNumber of video frames. Defaults to the model's own value (see the model list); each model has its own range
heightNoVideo height in pixels. Defaults to the model's own value (see the model list)
promptYesText prompt for video generation
ref_audiosNoReference audio clips (URL, data URI or base64), in order: ref_audios[0] is <Audio 1> (or <Audio 2> after a sounded reference video). Needs at least one ref_images or ref_videos entry
ref_imagesNoReference images (URL, data URI or base64), in order: ref_images[0] is <Picture 1> in the prompt, ref_images[1] is <Picture 2>, and so on. Each within a 1:4-4:1 aspect ratio. Only for models listing image refs in the model list; replaces first_frame_image
ref_videosNoReference videos (URL, data URI or base64), in order: ref_videos[0] is <Video 1>. Usually at most one, a few seconds long (see the model list). A video with a soundtrack takes <Audio 1> and shifts every ref_audios label by one
webhook_urlNoHTTPS URL for deAPI to call when this job finishes. Notifies your own systems; the tool still returns the result here
enhance_promptNoLet the API rewrite the prompt before generating. Only on models with a prompt-booster guide (422 otherwise); its fee is charged on the job and is not in the /price quote
guidance_scaleNoGuidance scale. Leave unset to use the model's own default; no model in the catalogue currently supports guidance
webhook_secretNo32-255 char key deAPI signs the callback with (HMAC-SHA256). Needs webhook_url. A secret given in chat stays in the transcript
negative_promptNoThings to exclude (optional)
last_frame_imageNoLast frame image (optional, URL, data URI, or base64). Not allowed with reference inputs
first_frame_imageNoFirst frame image as URL (e.g., from text_to_image result), data URI (data:image/png;base64,...), or base64 string. URLs are recommended when chaining from text_to_image to avoid base64 context bloat. Required unless ref_images or ref_videos are given; the two modes are mutually exclusive

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.6.3

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: mutual exclusivity of modes, that order of reference inputs changes tag meaning, that chat clients cannot pass files through, that enhance_prompt incurs an unquoted fee and 422s on unsupported models, and that webhook_secret persists in the transcript. These are non-obvious operational facts an agent could not infer from the schema alone. Return shape is covered by the output schema, so no gap there.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose, then cleanly sectioned into modes, input formats, and a prerequisite check; the worked prompt example is high-value rather than filler. A few lines duplicate the schema field descriptions almost verbatim (e.g. the tag ordering rules), which keeps it from being maximally tight.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 18-parameter generation tool with no annotations but a rich output schema, this covers mode selection, file transport constraints, tag semantics, and capability gating. It stops short of mentioning the pricing sibling (image_to_video_price) despite detailing an unquoted fee, and says nothing about job latency or polling beyond the webhook callback.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, and the description earns above baseline by explaining how ref_images/ref_videos/ref_audios map to <Picture N>/<Video N>/<Audio N> tags, giving a worked prompt example, and warning that swapping two images swaps what the tags refer to. Some of this repeats the per-field schema text, and it never explains webhook_url/seed/steps beyond what the schema already says, so it is not a full 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Generate video from static image(s), or from reference images, videos and audio') and immediately enumerates the two input modes with their exact field names. An agent can tell this apart from text_to_video and audio_to_video from the first line without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the two modes are mutually exclusive, describes which inputs select each mode, notes ref mode is only for models advertising 'refs: ...', and mandates a get_available_models() check before calling. It also frames the file-input decision (public URLs recommended, chaining with text_to_image) with a concrete when-to-use rationale.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.