Stage Face
stage_faceSynchronizes a voice recording's lip movements to a person's mouth at a chosen song time, generating mouth-shape tracks for lip sync in a text-driven DAW scene.
Instructions
Put a stretch of a voice recording on a person's mouth at a song time (lip sync): the mouth moves with the words while the song plays, on the take, Follow or rest pose the person shows, and in captures. The voice: audio= a recording (a sung vocal stem is best; wav, or webm/ogg with ffmpeg), or take= a take id (its own mic audio and the words said in it). start/end: the stretch, seconds on the recording's clock; at: the song second where start plays. words: [{word, start, end}] on the recording's clock (or a JSON file of them) shape the mouth per sound (closed lips on M B P, a pucker on OO); transcribe=True asks the speech server for them; with none, the jaw follows loudness only. engine: 'auto' uses Rhubarb Lip Sync when it is on this machine (it hears the mouth shapes in the voice itself, any language, singing too), else the words; 'rhubarb_english' its slower English recognizer guided by the words; 'words' the words only. The track is saved in /faces/ and listed in world.json faces (person -> [{file, at}]); a new track replaces the person's tracks it overlaps. The person's body needs mouth shapes (mhx export with morphs="mouth"); the page says face_unsupported when it has none. clear=True removes the person's tracks (file= only that one; the files stay). Replies with the track as text: a mouth shape letter and a jaw digit per 0.1 s (A closed, B teeth, C open, D wide, E round, F pucker, G lip under teeth, H L-tongue, X rest).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| at | No | ||
| end | No | ||
| file | No | ||
| take | No | ||
| audio | No | ||
| clear | No | ||
| scene | Yes | ||
| start | No | ||
| words | No | ||
| engine | No | auto | |
| person | Yes | ||
| scenes | No | ||
| transcribe | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |