runway_editVideo
Edit videos using text prompts and reference images through Runway's AI. Transform existing videos by describing desired changes, adjusting aspect ratios, and incorporating visual references for precise modifications.
Instructions
Edit a video using Runway Aleph. promptText is a prompt for the video. videoUri takes in a url of a video or a data uri of a video. Accepted Ratio values are 1280:720, 720:1280, 1104:832, 960:960, 832:1104, 1584:672, 848:480, 640:480. Use 1280:720 by default. It also accepts reference images, in the form of either a url or a base64 encoded image. Each reference image has a tag, which is a string that refers to the image from the user prompt. For example, if the user prompt is "IMG_1 on a red background", and the reference image has the tag "IMG_1", the model will use that reference image to generate the image.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| promptText | Yes | ||
| ratio | Yes | ||
| referenceImages | No | ||
| videoUri | Yes |
Implementation Reference
- src/index.ts:168-182 (handler)The handler function for the 'runway_editVideo' tool. It sends a POST request to Runway's /video_to_video endpoint using the 'gen4_aleph' model, with parameters for prompt text, video URI, ratio, and optional reference images. It awaits task completion and returns the task details as JSON.
async ({ promptText, videoUri, ratio, referenceImages }) => { const task = await callRunwayAsync("/video_to_video", { method: "POST", body: JSON.stringify({ promptText, videoUri, ratio, ...(referenceImages && referenceImages.length > 0 ? { references: referenceImages } : {}), model: "gen4_aleph", }), }); return { content: [{ type: "text", text: JSON.stringify(task) }] }; } - src/index.ts:160-167 (schema)Zod schema defining the input parameters for the 'runway_editVideo' tool: promptText (string), videoUri (string), ratio (string), and optional referenceImages array of objects with uri and optional tag.
{ promptText: z.string(), videoUri: z.string(), ratio: z.string(), referenceImages: z .array(z.object({ uri: z.string(), tag: z.string().optional() })) .optional(), }, - src/index.ts:155-183 (registration)Registration of the 'runway_editVideo' tool on the MCP server, including name, description, input schema, and handler function.
server.tool( "runway_editVideo", `Edit a video using Runway Aleph. promptText is a prompt for the video. videoUri takes in a url of a video or a data uri of a video. Accepted Ratio values are 1280:720, 720:1280, 1104:832, 960:960, 832:1104, 1584:672, 848:480, 640:480. Use 1280:720 by default. It also accepts reference images, in the form of either a url or a base64 encoded image. Each reference image has a tag, which is a string that refers to the image from the user prompt. For example, if the user prompt is "IMG_1 on a red background", and the reference image has the tag "IMG_1", the model will use that reference image to generate the image.`, { promptText: z.string(), videoUri: z.string(), ratio: z.string(), referenceImages: z .array(z.object({ uri: z.string(), tag: z.string().optional() })) .optional(), }, async ({ promptText, videoUri, ratio, referenceImages }) => { const task = await callRunwayAsync("/video_to_video", { method: "POST", body: JSON.stringify({ promptText, videoUri, ratio, ...(referenceImages && referenceImages.length > 0 ? { references: referenceImages } : {}), model: "gen4_aleph", }), }); return { content: [{ type: "text", text: JSON.stringify(task) }] }; } );