Make a person in a picture talk
lip_syncMake a person say a line, their mouth on the sound's words. mode still: a new video of the person in a picture (an image asset, or one frame of a video with atSeconds), in that picture's place and light, talking with natural head moves. mode dub: the person's own footage (a video asset, startSeconds to endSeconds) keeps everything as it was filmed and only the mouth changes, to dub new words or another language over what they said. The sound is a sound or video asset, or a part of one. With generate_speech's voiceAssetId, the person says new words in their own voice: pick a clean stretch of them speaking with read_transcript, make the line with generate_speech, then lip_sync them with it; edit_video can then change the new clip's world. The new video has the line's sound in it: place it as a video clip and do not place the sound again. A still is as long as the sound, at least 5 seconds, up to 60: a shorter sound is padded with silence to 5 seconds, where the person stays quiet, and the answer says where the words end, so the clip can be trimmed there. A dub is as long as the line: footage shorter than the line plays forward and back to cover it, so pick footage about as long as the line. It answers at once with a Job ID and the media is made in the background: start every generation you need, keep working, then call wait_for_jobs with all their Job IDs to get each Asset ID for patch_film. Uses credits for each second of the video: a still about 30 credits a second at standard and about 61 credits a second at high, 1.2 times that over 15 seconds, so a 5-second line uses 152 credits at standard; a dub about 51 credits a second of the longer of the footage and the line, so 5 seconds use 253 credits. Tell the person the cost before you start it.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | still: the person in a picture, an image or one frame of a video (atSeconds), comes to life saying the line. dub: a stretch of a video (startSeconds to endSeconds) keeps its real moves, face and framing, and only the mouth changes to say the line. | |
| assetId | Yes | still: an image asset, or a video asset with atSeconds for one of its frames: one face, turned to the camera, clear and not covered, in a picture from 2:5 (tall) to 5:2 (wide). dub: the video asset whose person says the line, with their face in view. Use the IDs search_assets gives. | |
| quality | Yes | still: standard (null) for most videos, made at 768p; high makes it at 1080p and uses twice the credits, only when the person asks for the sharpest picture. null with dub, which keeps the footage's own size. | |
| atSeconds | Yes | still with a video asset: the source second of the frame to animate, one where the face is clear and the mouth closed or still, which you checked with preview_frame or read_asset; null for an image, or for a video's first frame. null with dub. | |
| projectId | Yes | The project this call runs on without changing the open project. null (or leaving it out) uses the open project. wait_for_jobs takes the projectId of the calls that started the jobs. | |
| endSeconds | Yes | dub: source second of assetId where the footage to dub ends. null with still. | |
| audioAssetId | Yes | What they say: a sound asset, usually the speech generate_speech made (with voiceAssetId for the person's own voice), or a video or sound asset with speech. | |
| startSeconds | Yes | dub: source second of assetId where the footage to dub starts, with endSeconds: the part a clip plays, about as long as the line. null for both dubs the whole video, which is then 1 to 60 seconds long. null with still. | |
| audioEndSeconds | Yes | Source second of audioAssetId where the line ends. | |
| audioStartSeconds | Yes | Source second of audioAssetId where the line starts, with audioEndSeconds; null for both uses the whole asset. |