runway_generateVideo

Generate videos from images and text prompts with customizable aspect ratios and durations. Create 5-10 second videos using your images as a starting point for dynamic visual content.

Instructions

Generate a video from an image and a text prompt. Accepted ratios are 1280:720, 720:1280, 1104:832, 832:1104, 960:960, 1584:672. Use 1280:720 by default. For duration, there are only either 5 or 10 seconds. Use 5 seconds by default. If the user asks to generate a video, always first use generateImage to generate an image first, then use the image to generate a video.

Input Schema

TableJSON Schema

Name	Required	Description	Default
`duration`	Yes
`promptImage`	Yes
`promptText`	No
`ratio`	Yes

Implementation Reference

src/index.ts:84-96 (handler)

Handler function for the runway_generateVideo tool. It calls the Runway /image_to_video API endpoint with provided parameters and returns the task details as text content.

async (params) => {
  const task = await callRunwayAsync("/image_to_video", {
    method: "POST",
    body: JSON.stringify({
      model: "gen4_turbo",
      promptImage: params.promptImage,
      promptText: params.promptText,
      ratio: params.ratio,
      duration: params.duration,
    }),
  });
  return { content: [{ type: "text", text: JSON.stringify(task) }] };
}

src/index.ts:78-83 (schema)
Zod input schema defining parameters: promptImage (required string), promptText (optional string), ratio (string), duration (number).
```
{
  promptImage: z.string(),
  promptText: z.string().optional(),
  ratio: z.string(),
  duration: z.number(),
},
```

src/index.ts:75-97 (registration)

Registration of the runway_generateVideo tool using server.tool(), including name, description, input schema, and inline handler function.

server.tool(
  "runway_generateVideo",
  "Generate a video from an image and a text prompt. Accepted ratios are 1280:720, 720:1280, 1104:832, 832:1104, 960:960, 1584:672. Use 1280:720 by default. For duration, there are only either 5 or 10 seconds. Use 5 seconds by default. If the user asks to generate a video, always first use generateImage to generate an image first, then use the image to generate a video.",
  {
    promptImage: z.string(),
    promptText: z.string().optional(),
    ratio: z.string(),
    duration: z.number(),
  },
  async (params) => {
    const task = await callRunwayAsync("/image_to_video", {
      method: "POST",
      body: JSON.stringify({
        model: "gen4_turbo",
        promptImage: params.promptImage,
        promptText: params.promptText,
        ratio: params.ratio,
        duration: params.duration,
      }),
    });
    return { content: [{ type: "text", text: JSON.stringify(task) }] };
  }
);

src/index.ts:59-72 (helper)

Helper function used by the handler to perform asynchronous API calls to Runway, polling for task completion if a task ID is returned.

async function callRunwayAsync(
  path: string,
  opts: Partial<RequestInit> = {}
): Promise<RunwayTask> {
  const response = (await callRunway(path, opts)) as {
    id?: string;
  } & RunwayTask;
  // If the response has a taskId, wait for completion
  if (response?.id) {
    return waitForTaskCompletion(response.id);
  }
  // If no taskId, just return the response as is
  return response;
}

Runway API MCP Server