generate_video
Create videos from text descriptions or starting images using AI models. Specify prompts, duration, aspect ratio, and model parameters to generate custom video content.
Instructions
Generate videos from text prompts (text-to-video) or from images (image-to-video). Use list_models with category='video' to discover available models.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | Yes | Text description for the video (e.g., 'A slow-motion drone shot of Tokyo at night') | |
| image_url | No | Starting image URL for image-to-video models. Optional for text-to-video models. | |
| model | No | Model ID. Use 'fal-ai/kling-video/v2/master/text-to-video' for text-only, or image-to-video models like 'fal-ai/wan-i2v'. | fal-ai/wan-i2v |
| duration | No | Video duration in seconds | |
| aspect_ratio | No | Video aspect ratio (e.g., '16:9', '9:16', '1:1') | 16:9 |
| negative_prompt | No | What to avoid in the video (e.g., 'blur, distort, low quality') | |
| cfg_scale | No | Classifier-free guidance scale (0.0-1.0). Lower values give more creative results. |
Implementation Reference
- The core handler function for the 'generate_video' tool. It resolves the model ID, constructs parameters for the Fal.ai model, executes the generation using the queue strategy with timeout handling, and returns the generated video URL or an error message.async def handle_generate_video( arguments: Dict[str, Any], registry: ModelRegistry, queue_strategy: QueueStrategy, ) -> List[TextContent]: """Handle the generate_video tool.""" model_input = arguments.get("model", "fal-ai/wan-i2v") try: model_id = await registry.resolve_model_id(model_input) except ValueError as e: return [ TextContent( type="text", text=f"❌ {e}. Use list_models to see available options.", ) ] fal_args: Dict[str, Any] = { "prompt": arguments["prompt"], } # image_url is optional - only needed for image-to-video models if "image_url" in arguments: fal_args["image_url"] = arguments["image_url"] if "duration" in arguments: fal_args["duration"] = arguments["duration"] if "aspect_ratio" in arguments: fal_args["aspect_ratio"] = arguments["aspect_ratio"] if "negative_prompt" in arguments: fal_args["negative_prompt"] = arguments["negative_prompt"] if "cfg_scale" in arguments: fal_args["cfg_scale"] = arguments["cfg_scale"] # Use queue strategy with timeout protection for long-running video generation logger.info("Starting video generation with %s", model_id) try: video_result = await asyncio.wait_for( queue_strategy.execute(model_id, fal_args, timeout=180), timeout=185, # Slightly longer than internal timeout ) except asyncio.TimeoutError: return [ TextContent( type="text", text=f"❌ Video generation timed out after 180 seconds with {model_id}", ) ] if video_result is None: return [ TextContent( type="text", text=f"❌ Video generation failed or timed out with {model_id}", ) ] # Check for error in response if "error" in video_result: error_msg = video_result.get("error", "Unknown error") return [ TextContent( type="text", text=f"❌ Video generation failed: {error_msg}", ) ] # Extract video URL from result video_dict = video_result.get("video", {}) if isinstance(video_dict, dict): video_url = video_dict.get("url") else: video_url = video_result.get("url") if video_url: return [ TextContent( type="text", text=f"🎬 Video generated with {model_id}: {video_url}", ) ] return [ TextContent( type="text", text="❌ Video generation completed but no video URL was returned. Please try again.", ) ]
- The input schema and Tool definition for 'generate_video', defining parameters like prompt, optional image_url, model, duration, etc., with validation rules.Tool( name="generate_video", description="Generate videos from text prompts (text-to-video) or from images (image-to-video). Use list_models with category='video' to discover available models.", inputSchema={ "type": "object", "properties": { "prompt": { "type": "string", "description": "Text description for the video (e.g., 'A slow-motion drone shot of Tokyo at night')", }, "image_url": { "type": "string", "description": "Starting image URL for image-to-video models. Optional for text-to-video models.", }, "model": { "type": "string", "default": "fal-ai/wan-i2v", "description": "Model ID. Use 'fal-ai/kling-video/v2/master/text-to-video' for text-only, or image-to-video models like 'fal-ai/wan-i2v'.", }, "duration": { "type": "integer", "default": 5, "minimum": 2, "maximum": 10, "description": "Video duration in seconds", }, "aspect_ratio": { "type": "string", "default": "16:9", "description": "Video aspect ratio (e.g., '16:9', '9:16', '1:1')", }, "negative_prompt": { "type": "string", "description": "What to avoid in the video (e.g., 'blur, distort, low quality')", }, "cfg_scale": { "type": "number", "default": 0.5, "description": "Classifier-free guidance scale (0.0-1.0). Lower values give more creative results.", }, }, "required": ["prompt"], }, ),
- src/fal_mcp_server/server.py:61-85 (registration)Registration of the 'generate_video' handler in the TOOL_HANDLERS dictionary used by the MCP server to route tool calls to the appropriate handler function.TOOL_HANDLERS = { # Utility tools (no queue needed) "list_models": handle_list_models, "recommend_model": handle_recommend_model, "get_pricing": handle_get_pricing, "get_usage": handle_get_usage, "upload_file": handle_upload_file, # Image generation tools "generate_image": handle_generate_image, "generate_image_structured": handle_generate_image_structured, "generate_image_from_image": handle_generate_image_from_image, # Image editing tools "remove_background": handle_remove_background, "upscale_image": handle_upscale_image, "edit_image": handle_edit_image, "inpaint_image": handle_inpaint_image, "resize_image": handle_resize_image, "compose_images": handle_compose_images, # Video tools "generate_video": handle_generate_video, "generate_video_from_image": handle_generate_video_from_image, "generate_video_from_video": handle_generate_video_from_video, # Audio tools "generate_music": handle_generate_music, }