image_to_video
Convert an image to video with text prompts using MiniMax AI. Input the first frame image and a description to generate a dynamic video output.
Instructions
Generate a video based on an image.
Note: This tool calls MiniMax API and may incur costs. Use only when explicitly requested by the user.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| asyncMode | No | Whether to use async mode. Defaults to False. If True, the video generation task will be submitted asynchronously and the response will return a task_id. Should use `query_video_generation` tool to check the status of the task and get the result. | |
| firstFrameImage | Yes | Path to the first frame image | |
| model | No | Model to use, values: ["I2V-01", "I2V-01-Director", "I2V-01-live"] | I2V-01 |
| outputDirectory | No | The directory to save the output file. `outputDirectory` is relative to `MINIMAX_MCP_BASE_PATH` (or `basePath` in config). The final save path is `${basePath}/${outputDirectory}`. For example, if `MINIMAX_MCP_BASE_PATH=~/Desktop` and `outputDirectory=workspace`, the output will be saved to `~/Desktop/workspace/` | |
| outputFile | No | Path to save the generated video file, automatically generated if not provided | |
| prompt | Yes | Text prompt for video generation |
Implementation Reference
- src/mcp-server.ts:555-631 (registration)Registration of the 'image_to_video' MCP tool including input schema (Zod) and execution handler that calls VideoAPI.generateVideo
private registerImageToVideoTool(): void { this.server.tool( 'image_to_video', 'Generate a video based on an image.\n\nNote: This tool calls MiniMax API and may incur costs. Use only when explicitly requested by the user.', { model: z .string() .optional() .default('I2V-01') .describe('Model to use, values: ["I2V-01", "I2V-01-Director", "I2V-01-live"]'), prompt: z.string().describe('Text prompt for video generation'), firstFrameImage: z.string().describe('Path to the first frame image'), outputDirectory: COMMON_PARAMETERS_SCHEMA.outputDirectory, outputFile: z .string() .optional() .describe('Path to save the generated video file, automatically generated if not provided'), asyncMode: z .boolean() .optional() .default(false) .describe('Whether to use async mode. Defaults to False. If True, the video generation task will be submitted asynchronously and the response will return a task_id. Should use `query_video_generation` tool to check the status of the task and get the result.'), }, async (params) => { try { // If no output filename is provided, generate one automatically if (!params.outputFile) { const promptPrefix = params.prompt.substring(0, 20).replace(/[^\w]/g, '_'); params.outputFile = `i2v_${promptPrefix}_${Date.now()}`; } const result = await this.videoApi.generateVideo(params); if (params.asyncMode) { return { content: [ { type: 'text', text: `Success. Video generation task submitted: Task ID: ${result.task_id}. Please use \`query_video_generation\` tool to check the status of the task and get the result.`, }, ], }; } // Handle different output formats if (this.config.resourceMode === RESOURCE_MODE_URL) { return { content: [ { type: 'text', text: `Success. Video URL: ${result.video_url}`, }, ], }; } else { return { content: [ { type: 'text', text: `Video saved: ${result.video_path}`, }, ], }; } } catch (error) { return { content: [ { type: 'text', text: `Failed to generate video: ${error instanceof Error ? error.message : String(error)}`, }, ], }; } } ); } - src/mcp-rest-server.ts:353-375 (schema)JSON schema definition for 'image_to_video' tool in REST server's listTools handler
name: 'image_to_video', description: 'Generate video based on image', arguments: [ { name: 'prompt', description: 'Text prompt for video generation', required: true }, { name: 'firstFrameImage', description: 'Path to first frame image', required: true }, { name: 'model', description: 'Model to use, values: ["I2V-01", "I2V-01-Director", "I2V-01-live"]', required: false }, { name: 'outputDirectory', description: OUTPUT_DIRECTORY_DESCRIPTION, required: false }, { name: 'outputFile', description: 'Output file path, auto-generated if not provided', required: false }, { name: 'async_mode', description: 'Whether to use async mode. Defaults to False. If True, the video generation task will be submitted asynchronously and the response will return a task_id. Should use `query_video_generation` tool to check the status of the task and get the result', required: false } ], inputSchema: { type: 'object', properties: { prompt: { type: 'string' }, firstFrameImage: { type: 'string' }, model: { type: 'string' }, outputDirectory: { type: 'string' }, outputFile: { type: 'string' }, async_mode: { type: 'boolean' } }, required: ['prompt', 'firstFrameImage'] } }, - src/mcp-rest-server.ts:622-652 (handler)Handler method for 'image_to_video' tool in REST server that validates params and delegates to MediaService.generateVideo with retry logic
private async handleImageToVideo(args: any, api: MiniMaxAPI, mediaService: MediaService, attempt = 1): Promise<any> { try { // Ensure model is suitable for image to video conversion if (!args.model) { args.model = 'I2V-01'; } // Ensure firstFrameImage parameter exists if (!args.firstFrameImage) { throw new Error('Missing required parameter: firstFrameImage'); } // Auto-generate output filename if not provided if (!args.outputFile) { const promptPrefix = args.prompt.substring(0, 20).replace(/[^\w]/g, '_'); args.outputFile = `i2v_${promptPrefix}_${Date.now()}`; } // Call media service to handle request const result = await mediaService.generateVideo(args); return result; } catch (error) { if (attempt < MAX_RETRY_ATTEMPTS) { // console.warn(`[${new Date().toISOString()}] Failed to generate video, attempting retry (${attempt}/${MAX_RETRY_ATTEMPTS})`, error); // Delay retry await new Promise(resolve => setTimeout(resolve, RETRY_DELAY * Math.pow(2, attempt - 1))); return this.handleImageToVideo(args, api, mediaService, attempt + 1); } throw this.wrapError('Failed to generate video', error); } } - MediaService.generateVideo helper method used by image_to_video handlers, formats response and calls VideoAPI
public async generateVideo(params: any): Promise<any> { this.checkInitialized(); try { // Auto-generate output filename if not provided if (!params.outputFile) { const promptPrefix = params.prompt.substring(0, 20).replace(/[^\w]/g, '_'); params.outputFile = `video_${promptPrefix}_${Date.now()}`; } const result = await this.videoApi.generateVideo(params); if (params.async_mode) { return { content: [ { type: 'text', text: `Success. Video generation task submitted: Task ID: ${result.task_id}. Please use \`query_video_generation\` tool to check the status of the task and get the result.`, }, ], }; } else if (this.config.resourceMode === RESOURCE_MODE_URL) { return { content: [ { type: 'text', text: `Success. Video URL: ${result.video_url}`, }, ], }; } else { return { content: [ { type: 'text', text: `Success. Video saved as: ${result.video_path}`, }, ], }; } } catch (error) { // console.error(`[${new Date().toISOString()}] Failed to generate video:`, error); throw this.wrapError('Failed to generate video', error); } } - src/api/video.ts:17-149 (handler)Core handler in VideoAPI for generating video from image+text (image_to_video). Submits to MiniMax API /v1/video_generation (supports first_frame_image), polls status, downloads/saves video file or returns URL.
async generateVideo(request: VideoGenerationRequest): Promise<any> { // Validate required parameters if (!request.prompt || request.prompt.trim() === '') { throw new MinimaxRequestError(ERROR_PROMPT_REQUIRED); } try { // Ensure model is valid const model = this.ensureValidModel(request.model); // Prepare request data const requestData: Record<string, any> = { model: model, prompt: request.prompt }; // Process first frame image if (request.firstFrameImage) { // Check if it's a URL or data URL if (!request.firstFrameImage.startsWith(('http://')) && !request.firstFrameImage.startsWith(('https://')) && !request.firstFrameImage.startsWith(('data:'))) { // If it's a local file, convert to data URL if (!fs.existsSync(request.firstFrameImage)) { throw new MinimaxRequestError(`First frame image file does not exist: ${request.firstFrameImage}`); } const imageData = fs.readFileSync(request.firstFrameImage); const base64Image = imageData.toString('base64'); requestData.first_frame_image = `data:image/jpeg;base64,${base64Image}`; } else { requestData.first_frame_image = request.firstFrameImage; } } // Process resolution if (request.resolution) { requestData.resolution = request.resolution; } // Process duration if (request.duration) { requestData.duration = request.duration; } // Step 1: Submit video generation task const response = await this.api.post<any>('/v1/video_generation', requestData); // Get task ID const taskId = response?.task_id; if (!taskId) { throw new MinimaxRequestError('Unable to get task ID from response'); } if (request.asyncMode) { return { task_id: taskId, } } // Step 2: Wait for video generation task to complete let fileId: string | null = null; const maxRetries = model === "MiniMax-Hailuo-02" ? 60 : 30; // Maximum 30 attempts, total duration 10 minutes (30 * 20 seconds). MiniMax-Hailuo-02 model has a longer processing time, so we need to wait for a longer time const retryInterval = 20; // 20 second interval for (let attempt = 0; attempt < maxRetries; attempt++) { // Query task status const statusResponse = await this.api.get<any>(`/v1/query/video_generation?task_id=${taskId}`); const status = statusResponse?.status; if (status === 'Fail') { throw new MinimaxRequestError(`Video generation task failed, task ID: ${taskId}`); } else if (status === 'Success') { fileId = statusResponse?.file_id; if (fileId) { break; } throw new MinimaxRequestError(`File ID missing in success response, task ID: ${taskId}`); } // Task still processing, wait and retry await new Promise(resolve => setTimeout(resolve, retryInterval * 1000)); } if (!fileId) { throw new MinimaxRequestError(`Failed to get file ID, task ID: ${taskId}`); } // Step 3: Get video result const fileResponse = await this.api.get<any>(`/v1/files/retrieve?file_id=${fileId}`); const downloadUrl = fileResponse?.file?.download_url; if (!downloadUrl) { throw new MinimaxRequestError(`Unable to get download URL for file ID: ${fileId}`); } // If URL mode, return URL directly const resourceMode = this.api.getResourceMode(); if (resourceMode === RESOURCE_MODE_URL) { return { video_url: downloadUrl, task_id: taskId, }; } // Step 4: Download and save video const outputPath = buildOutputFile(`video_${taskId}`, request.outputDirectory, 'mp4', true); try { const videoResponse = await requests.default.get(downloadUrl, { responseType: 'arraybuffer' }); // Ensure directory exists const dirPath = path.dirname(outputPath); if (!fs.existsSync(dirPath)) { fs.mkdirSync(dirPath, { recursive: true }); } // Save file fs.writeFileSync(outputPath, Buffer.from(videoResponse.data)); return { video_path: outputPath, task_id: taskId, } } catch (error) { throw new MinimaxRequestError(`Failed to download or save video: ${String(error)}`); } } catch (error) { if (error instanceof MinimaxRequestError) { throw error; } throw new MinimaxRequestError(`Unexpected error occurred during video generation: ${String(error)}`); } }