runway_generateVideo
Generate videos from images and text prompts with customizable aspect ratios and durations. Create 5-10 second videos using your images as a starting point for dynamic visual content.
Instructions
Generate a video from an image and a text prompt. Accepted ratios are 1280:720, 720:1280, 1104:832, 832:1104, 960:960, 1584:672. Use 1280:720 by default. For duration, there are only either 5 or 10 seconds. Use 5 seconds by default. If the user asks to generate a video, always first use generateImage to generate an image first, then use the image to generate a video.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| duration | Yes | ||
| promptImage | Yes | ||
| promptText | No | ||
| ratio | Yes |
Implementation Reference
- src/index.ts:84-96 (handler)Handler function for the runway_generateVideo tool. It calls the Runway /image_to_video API endpoint with provided parameters and returns the task details as text content.
async (params) => { const task = await callRunwayAsync("/image_to_video", { method: "POST", body: JSON.stringify({ model: "gen4_turbo", promptImage: params.promptImage, promptText: params.promptText, ratio: params.ratio, duration: params.duration, }), }); return { content: [{ type: "text", text: JSON.stringify(task) }] }; } - src/index.ts:78-83 (schema)Zod input schema defining parameters: promptImage (required string), promptText (optional string), ratio (string), duration (number).
{ promptImage: z.string(), promptText: z.string().optional(), ratio: z.string(), duration: z.number(), }, - src/index.ts:75-97 (registration)Registration of the runway_generateVideo tool using server.tool(), including name, description, input schema, and inline handler function.
server.tool( "runway_generateVideo", "Generate a video from an image and a text prompt. Accepted ratios are 1280:720, 720:1280, 1104:832, 832:1104, 960:960, 1584:672. Use 1280:720 by default. For duration, there are only either 5 or 10 seconds. Use 5 seconds by default. If the user asks to generate a video, always first use generateImage to generate an image first, then use the image to generate a video.", { promptImage: z.string(), promptText: z.string().optional(), ratio: z.string(), duration: z.number(), }, async (params) => { const task = await callRunwayAsync("/image_to_video", { method: "POST", body: JSON.stringify({ model: "gen4_turbo", promptImage: params.promptImage, promptText: params.promptText, ratio: params.ratio, duration: params.duration, }), }); return { content: [{ type: "text", text: JSON.stringify(task) }] }; } ); - src/index.ts:59-72 (helper)Helper function used by the handler to perform asynchronous API calls to Runway, polling for task completion if a task ID is returned.
async function callRunwayAsync( path: string, opts: Partial<RequestInit> = {} ): Promise<RunwayTask> { const response = (await callRunway(path, opts)) as { id?: string; } & RunwayTask; // If the response has a taskId, wait for completion if (response?.id) { return waitForTaskCompletion(response.id); } // If no taskId, just return the response as is return response; }