kling_lip_sync
Replace a video's lip movements with audio-driven or text-to-speech speech for dubbing or talking-head videos.
Instructions
Synchronize lip movements in a video to match a given audio track or text.
Takes an existing video (by URL or task ID) and replaces the speaker's lip
movements so they match the provided audio. In 'text2video' mode the audio
is generated from the supplied text via TTS.
Use this when:
- You want to dub a video into a different language
- You want to replace the audio of a generated video with custom speech
- You want to create a talking-head video from text
Returns:
Task ID and lip-sync video information.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | Lip-sync mode. 'audio2video' to drive lips from an audio file or URL; 'text2video' to generate speech from text and drive the video. | |
| text | No | Text to convert to speech. Required when mode='text2video'. | |
| video_id | No | Task ID of a previously generated video to use as the source. Provide either video_url or video_id. | |
| voice_id | No | Voice ID to use for text-to-speech synthesis (mode='text2video'). | |
| audio_url | No | URL of the driving audio. Required when mode='audio2video' and audio_type='url'. | |
| video_url | No | URL of the source video whose lip movements will be replaced. Provide either video_url or video_id. | |
| audio_file | No | Base64-encoded audio file content. Required when mode='audio2video' and audio_type='file'. | |
| audio_type | No | Audio source type. 'url' (default) to supply audio_url; 'file' to supply audio_file as a base64-encoded string. | url |
| voice_speed | No | Speech speed multiplier (default 1.0). Used when mode='text2video'. | |
| callback_url | No | Webhook URL that receives a POST when the lip-sync task completes. | |
| voice_language | No | Language of the TTS voice. 'zh' for Chinese (default), 'en' for English. Used when mode='text2video'. | zh |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |