Audio To Video
audio_to_videoGenerates video conditioned on an audio file and text prompt, producing synchronized audio-visual output with optional first and last frame anchor images.
Instructions
Generate video from audio and text prompt.
Creates videos conditioned on audio content. The audio drives the video generation, allowing synchronized audio-visual output. Optionally accepts anchor frame images.
IMPORTANT: Check model specifications using get_available_models() before calling. Pay attention to features.supports_guidance, features.supports_steps, and model limits.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"prompt": "<prompt>",
"model": "<model>",
"audio": "<audio>"
},
"name": "audio_to_video"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| fps | No | Frames per second. Defaults to the model's own value (see the model list); most models fix it | |
| seed | No | Random seed (-1 for random) | |
| audio | Yes | Audio file as URL, data URI (data:audio/mpeg;base64,...), or base64 string. Conditions the video generation on the audio content | |
| model | Yes | Video generation model name (must support audio2video inference type) | |
| steps | No | Inference steps. Only used if model's features.supports_steps is true. Limits per model (see the model list) | |
| width | No | Video width in pixels. Defaults to the model's own value (see the model list) | |
| frames | No | Number of video frames. Defaults to the model's own value (see the model list); each model has its own range | |
| height | No | Video height in pixels. Defaults to the model's own value (see the model list) | |
| prompt | Yes | Text prompt for video generation | |
| guidance | No | Guidance scale. Only used if model's features.supports_guidance is true. Limits per model (see the model list) | |
| webhook_url | No | HTTPS URL for deAPI to call when this job finishes. Notifies your own systems; the tool still returns the result here | |
| webhook_secret | No | 32-255 char key deAPI signs the callback with (HMAC-SHA256). Needs webhook_url. A secret given in chat stays in the transcript | |
| negative_prompt | No | Things to exclude (optional) | |
| last_frame_image | No | Last frame anchor image (URL, data URI, or base64, optional) | |
| first_frame_image | No | First frame anchor image (URL, data URI, or base64, optional) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||