create timeline lipsync
lipsync_create-timelineLipsyncs a saved video with multiple audio clips placed on separate timeline ranges, letting you dub only selected shots or switch audio or speakers between segments.
Instructions
Lipsync one saved video with several audio clips, each on its own stretch of the video like a Studio timeline: to lipsync only some ranges, or to switch audio or speaker between shots. For one audio across the whole video use lipsync_create; to translate and dub a whole video use dubbing_create when available.
Each lipsynced clip takes exactly one of audioAssetId or audioUrl, a span in seconds or in frames (not a mix), and optionally audioStartTime and audioEndTime to play part of that audio. Clips may come in any order but must not overlap; parts no clip covers keep their original picture and, unless outputAudio plays, their original sound. Frame spans need the video's fps: read it from assets_get-playback-metadata when it returns one (signed-in users only) and never guess it; otherwise use seconds. Get speaker.faceTrackUrl from transcriptions_slice-speaker-face-track when available.
To lipsync only the speaking shots of a dub, pass the full dubbed track as outputAudio, give speaking shots clips that play their part of it, and mark the other shots lipsync: false. outputAudio replaces the sound of the whole output; if it is omitted while a clip sets lipsync: false, the one audio the other clips share plays instead, so pass it whenever they use several.
Before anything is created, the tool validates the input (options included), spans, audio ranges and overlaps, and per-clip errors name the clip index. Overlaps are compared only within one unit, so use one unit for every clip. Problems only the API detects, such as an unknown audio asset or a speaker point off the face, surface after a new project may already exist.
Paid generation, sync-3 unless model is set. Estimate and confirm before creating; this starts an asynchronous generation and returns its id and status. Poll that id with generations_get and wait: true.
Without projectId it first creates a NEW project, named projectName or a default; default new work to that. Pass projectId with projectIntent=explicit only when the user explicitly asks to work in that project, selects it in the interface, or it was just created for this request; a matching workflow or name or an open editor is never permission to modify it. Send a fresh idempotencyKey, plus a fresh projectCreationKey when there is no projectId, and after a lost or uncertain response retry only with identical arguments and the same keys. Never resend a request that had no idempotencyKey: it can start a second paid generation.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| clips | Yes | Clips placed on the video, in any order, without overlapping. Parts no clip covers keep their original picture, and their original sound unless outputAudio plays; clips with lipsync: false keep their picture but play outputAudio. | |
| model | No | Model to use. Defaults to sync-3. | |
| options | No | Optional model controls. Omit any field to use its default. | |
| projectId | No | Existing Sync project id, only for a project the user explicitly selected or one just created for this request. | |
| outputAudio | No | One audio that plays over the whole finished output from its first second, replacing the sound of every part, such as the full dubbed track. Defaults, when a clip sets lipsync: false, to the only audio the clips use. | |
| projectName | No | Name for a NEW project only. Never search or reuse a matching name. Cannot be combined with projectId. | |
| videoAssetId | Yes | Sync asset id of the video. | |
| projectIntent | No | Required with projectId. Set only for an explicit user project selection/update, or the new project just created for this request. Never infer intent from a matching project name, workflow name or active editor. | |
| idempotencyKey | No | Unique key for this generation. Reuse it with identical arguments when retrying after a lost response. Passed as the Idempotency-Key header. | |
| organizationId | No | Run only this call in another of the user’s Sync organizations, by id from organizations_list, when the user asks for it (for example “generate this in the Acme org”). The session keeps its organization; reads of the ids this call returns stay in this organization. Omit to use the session’s organization. | |
| projectCreationKey | No | Fresh random UUID identifying this NEW project creation, not a project ID. Required for keyed generation without projectId. Persist this key, name and generation key on retries; the server derives a separate project UUID. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||