Skip to main content
Glama

Generate Video with Replicate

replicate_generate_video

Generates a video clip from a text prompt and optional starting image using Replicate models, returning openable URLs and optionally downloading the MP4 locally.

Instructions

Generate a video clip from a text prompt (and optionally a starting image). Video generation is slow — typically 1-5 minutes per clip.

DISPLAY REQUIREMENT — after this tool returns successfully, include the URL(s) printed in the tool's text content so the user can open the video. URLs expire in ~24h.

Args:

  • prompt (string): Text description of the video.

  • model (string, default "kling-pro"): Curated key (kling-pro, minimax-video, hunyuan-video, luma-ray, wan-2.2, grok-video, seedance) or "owner/name[:version]".

  • image_url (string, optional): Starting frame for image-to-video. Not all models support this.

  • duration_seconds (1-60, optional): Desired duration. Model-dependent.

  • aspect_ratio ("16:9" | "9:16" | "1:1", optional): Aspect ratio.

  • extra_input (object, optional): Additional model-specific inputs.

  • download (boolean, default true): Download the MP4 locally.

  • timeout_ms: Max wait. Default 300000 (5min). For very long videos, increase or rely on the pending+poll flow.

Returns: PredictionResult (see replicate_generate_image for shape). The local_paths will contain .mp4 files when downloaded.

Tip: If timeout_ms is exceeded, the result will have pending=true and a prediction_id. Wait a minute, then call replicate_get_prediction.

Input Schema

NameRequiredDescriptionDefault
modelNoEither a curated key (kling-pro, minimax-video, hunyuan-video, luma-ray, wan-2.2, grok-video, seedance) or a Replicate identifier.kling-pro
promptYesText prompt describing the video.
downloadNoWhether to download the generated files locally. Default true. When false, only Replicate URLs are returned (URLs expire after ~24h).
image_urlNoOptional starting image URL for image-to-video. Not all models support this — check model schema.
timeout_msNoMax ms to wait for the prediction. If exceeded, returns the prediction ID so you can poll via replicate_get_prediction. Default: 300000 (5min).
extra_inputNoAdditional model-specific inputs.
aspect_ratioNoAspect ratio.
duration_secondsNoDesired duration in seconds. Model-dependent.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv3.3.1
    • addedInput schema / properties / extra_input / propertyNames
      Added value: +{
      +  "type": "string"
      +}
  2. First observedv3.2.3

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (not read-only, open-world, non-idempotent); the description adds the traits that actually matter here — multi-minute latency, ~24h URL expiry, local MP4 download behavior, and the timeout/pending fallback with the follow-up tool named. It also carries an explicit post-success display requirement for the user-facing URL.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and latency, then the display requirement, then args and the polling tip. The Args list duplicates the schema field-by-field and could be trimmed, but the total length is justified for an 8-parameter tool with a nontrivial async flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does the necessary work: it names the return type, points to replicate_generate_image for shape, and explains that local_paths holds .mp4 files when download=true. That cross-tool indirection for the return shape is the one remaining soft spot, but an agent can still call and interpret results correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is already 100%, including the enum, defaults, min/max and the extra_input object, so the Args block largely restates structured data. The only genuine additions — curated model keys, 'not all models support image_url', 'model-dependent' duration, and the advice to raise timeout_ms for long videos — are already substantially present in the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — generate a video clip from a text prompt, optionally image-to-video — and the medium immediately distinguishes it from replicate_generate_image, replicate_generate_audio and replicate_generate_3d. An agent knows what it produces without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: it is slow (1-5 min), image_url is only usable with supporting models, and when timeout_ms is exceeded the pending=true + replicate_get_prediction polling path applies. It stops short of explicitly ruling out alternatives such as replicate_run_model or replicate_generate_image, so it is not a full when/when-not statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.