generate_video
Generate a playable MP4 video from a text prompt. Converts raw frames and optional soundtrack into an H.264 file with AAC audio for playback anywhere.
Instructions
Generate a short video from a text prompt; returns a local MP4 path.
mlx-serve answers with raw frames (+ optional soundtrack); this tool encodes them into an H.264 MP4 (AAC audio when present) locally via ffmpeg, so the returned file is playable anywhere. Generation is slow — minutes per clip depending on frame count and resolution.
Args: prompt: Scene description for the video. model: Video model id. Defaults to MLX_SERVE_VIDEO_MODEL. num_frames: Frame count. LTX backends use an 8N+1 ladder (default 9; e.g. 9/25/33/49/57/81...); MiniMax-H3 uses 17k+5 (default 56). width: Pixel width (defaults: LTX 384, H3 256). Two-stage pipelines need both dimensions divisible by 64. height: Pixel height (defaults: LTX/H3 256). steps: Sampling steps (backend-specific defaults). turbo: Use the distilled 4-step turbo path where the model pack provides it. seed: Optional seed. cfg_scale: Guidance scale. first_frame_image_path: Optional local image to condition the first frame. last_frame_image_path: Optional local image to condition the final frame. audio_path: Optional local WAV to mix as soundtrack (must match frame duration).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| seed | No | ||
| model | No | ||
| steps | No | ||
| turbo | No | ||
| width | No | ||
| height | No | ||
| prompt | Yes | ||
| cfg_scale | No | ||
| audio_path | No | ||
| num_frames | No | ||
| last_frame_image_path | No | ||
| first_frame_image_path | No |