Motion from a Clip
lab_reference_motionRender a new clip FROM the motion of an existing one: your reference clip leads (camera move, blocking, timing), the model restyles what happens in it. Pass the clip's URL directly — the one you already have from lab_list_assets (asset_type 'video'), lab_job_status or lab_review_queue; nothing is looked up. Reference images bind cast, place and look; in_sec/out_sec cut the reference server-side before it ever reaches the provider, so you pay for the excerpt, not the source. Same backend path as lab_animate's source_video_id, with the fields that only exist there. Costs credits when called without get_cost. WITHOUT a reference clip (leave reference_video_url out), the images and sounds in referenzen alone guide the run — only for models whose lab_list_models entry says referenzen_ohne_video: true, and only FACELESS references (ort, stil, bild, stimme, ton): Seedance 2.5 rejects faces as reference images, even invented characters (measured 2026-09-27), so figur is refused before anything is charged. For a person, build a still with them first and animate it with lab_animate. Name the references in the prompt as 'reference only, never shown on screen', and avoid the word 'static' for short shots — it tends to return frozen frames.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ton | No | Which sound the result carries. quelle: the reference clip's own audio is laid back under the new picture after generation — the way to keep a spoken take, since models that regenerate audio replace the voice. generiert: keep the model's sound (needs generate_audio: true). keiner: no sound. Omitted: quelle when a reference clip is given and generate_audio is off, generiert when generate_audio is on, keiner otherwise. The finished run reports in metadata what actually lies underneath (ton) and how far picture and sound differ in length (ton_drift_s). | |
| task | No | reference (default): the references guide a new clip. editing: the picture of the reference clip stays and only what the prompt names changes — e.g. 'keep the picture, generate matching sound effects, no music' with generate_audio: true. Needs a reference clip. Edit verbs in the prompt are expected here, not refused. | |
| tool | No | Video model id — copy it verbatim from lab_list_models (kind: video), and only models the catalog marks v2v take a reference clip at all. Omit to let the backend use its own default for this path. Durations, resolutions and price live in the catalog, this text names none. | |
| draft | No | Render a cheap DRAFT first (Seedance 2.5 and wan_vace only — other models refuse it before anything is charged). On Seedance 2.5 the draft is 480p, priced at that tier, and carries a draft_id that stays valid for 7 days and only for the account that ordered it. Finish an accepted draft in 1080p ONLY with lab_fertigstellen — never by downloading and re-uploading it: that is a new run with new motion, not the draft. Any `resolution` you pass is ignored for a draft, and the answer says so. | |
| in_sec | No | Cut the reference clip server-side before it reaches the provider: start of the excerpt, in seconds from the clip's beginning. Only together with out_sec. | |
| prompt | Yes | What the new clip should show. Describe it — do NOT phrase it as an instruction to edit or extend the reference; some models classify the run by the verb and then reject the aspect ratio and duration after queueing. | |
| out_sec | No | End of the excerpt in the reference clip, in seconds. Must lie behind in_sec; both are set together or neither. | |
| get_cost | No | Preflight: return the price in credits and your balance WITHOUT generating and without spending anything, reading the same catalog as lab_list_models. Do this before a long take or when the user should confirm the cost. | |
| use_depth | No | Condition the run on a depth pass of the reference clip. Defaults to ON (true = 'voll': depth of the whole frame); pass false to opt out. Models that do not have a depth stage ignore it — the answer says so rather than pretending it happened. 'personen': depth ONLY on the people, the original footage around them — the model stops copying build, hair and clothing of the original performers and the room keeps its colours. With 'personen' the run PAUSES after the mask: only the mask (SAM) is charged, the answer carries cost (mask) and cost_lauf (the run), and the clip waits as wartet_auf_freigabe until the human has looked at the preview (metadata.personen_referenz_url) and you call lab_freigeben. Needs a reference clip and a model with a depth stage; not together with face masks or task='editing'. To RE-FILM a take from new camera angles, pass false: a depth pass locks the geometry of the reference, and the new angles cannot happen. task='editing' ignores it anyway. | |
| kamerapfad | No | The camera move as INPUT instead of prose: an object following shared/kamerapfad/schema.json — poses over time (position, look-at or quaternion, metres, Y up) from the Kadrage terrain flight or drawn by hand, or motion phases as lab_kamerafahrt reads them from a clip (its answer carries a ready `kamerapfad`). The backend validates it, derives phases, and adds the move as a paragraph after your prompt, before the reference markers (the reference clip still leads the motion as the first video reference). WHICH FORM that paragraph takes depends on the model: prose for most families, bracketed director commands for the MiniMax/Hailuo family, which parses them. NO MODEL RECEIVES THE POSES TODAY — the answer says which form went out (kamerapfad.angewandt = als_text or als_regiebefehle), names the family rule and the source it rests on (kamerapfad.sprache), and lists what that form could not carry. If your own prompt already carries bracketed commands, the path is NOT sent on top of them (angewandt = eigene_regie, with kamerapfad.eigene_regiebefehle naming what it found). A `zusammenfassung` inside the path is display only and is NEVER sent — the move is always rebuilt from the phases; if nothing can be built, the answer says angewandt = nicht_ableitbar with the reason, and nothing of the path is in the prompt. Mutually exclusive with kamerapfad_url. | |
| project_id | Yes | Project id (from lab_list_projects) — positive integer as string. On this path it travels in the request BODY, not in the URL, but it is just as required: the new clip is filed under this project. | |
| referenzen | No | The typed successor of character_ref_urls: each reference says WHAT it is, so a place or a look no longer has to travel disguised as a character. Since 2026-09-03 it carries all three channels, not just images: motion references ride alongside the reference clip, sound references go into the model's audio slots. The position IS the address — the images are @Image1..n, the reference clip is @Video1 and the extra motion references follow it, the sounds are @Audio1..n. Image references are MEASURED before they are sent and scaled up to the model's minimum if they fall below it; a reference is never rejected for being small. Mutually exclusive with character_ref_urls — two sources for one list means one of them silently loses. What a model takes per channel is in lab_list_models (referenz_grenzen). | |
| resolution | No | Render resolution on this path, copied verbatim from the model's resolutions in lab_list_models. The run is priced at the resolution you choose — the low tier costs a fraction of the default — so pass it for sound-only or test runs. Omitted, the model's default applies. | |
| wechsel_von | No | The model id this call switches AWAY from (as listed in lab_list_models), with the same references as before. The answer then carries `wechsel`: which references the switch cuts off and why, and which ones the new model picks up. Nothing is sent to the old model; the field only names the comparison. | |
| aspect_ratio | No | Output aspect ratio on this path, written as width:height or 'auto'. Only models whose lab_list_models entry takes it on the reference path accept it; for any other model the backend refuses it before anything is charged, and the refusal lists the allowed values. Omitted, the model's fixed default applies. | |
| ton_fuehrung | No | What the model HEARS as guidance while it renders — not what the result carries (that is `ton`). voll (default, same as omitting it): unchanged behaviour. stimme: the backend splits the reference excerpt (after in_sec/out_sec) with demucs and sends ONLY the vocal stem, unchanged, as the single audio reference @Audio1 — so a speaking or singing performer moves the lips without the music bed dragging along. The stem is never pitch-shifted or altered in any way. The demucs split is charged per second of the excerpt and shows up in get_cost (ton_fuehrung.cost); it is cached per clip and in/out, a repeat costs nothing. Needs a reference clip and a model with an audio slot; not together with sound references in `referenzen`. keine: no audio reference at all. With ton='quelle' the original mix of the clip — not the stem — is laid back under the result. | |
| generate_audio | No | Let the model generate sound with the clip — only models whose lab_list_models entry can output audio take it on this path. Omitted = no generated sound. Generated audio REPLACES any voice in the reference — to keep a spoken take, leave it off. | |
| kamerapfad_url | No | Address of a kamerapfad JSON instead of the object — the url of a file in THIS project (lab_list_assets), e.g. the kamerapfad JSON that Kadrage writes next to its flight video. Fetched by the backend under the same ownership rule as every reference: a file of another project is a named rejection before anything is charged. Mutually exclusive with kamerapfad. | |
| last_frame_url | No | Guiding frame for the END of the new clip (image URL). Same rule as first_frame_url: wan_vace only, rejected elsewhere rather than dropped. | |
| first_frame_url | No | Guiding frame for the START of the new clip (image URL). Only the wan_vace path binds guiding frames to the provider — with any other model the backend REJECTS this field instead of swallowing it, because a swallowed guiding frame means paying for a run identical to the one without it. | |
| duration_seconds | No | Length of the NEW clip. Must lie inside the chosen model's duration grid from lab_list_models — the per-model ceiling is enforced by the backend and rejected rather than silently clamped. | |
| raum_beschreiben | No | With depth on, a vision call describes ONLY the room of a still from the reference (place, light, colours, materials — no people) and adds it as a paragraph to the prompt, so a depth run does not come back grey. ON by default; false switches it off. The setting is remembered per project — omit it to use the project's last choice. Skipped automatically when an `ort` reference is given. The answer reports raum.angewandt and the text; metadata.prompt_gesendet holds the prompt as sent. | |
| rechte_bestaetigt | No | REQUIRED when the reference clip or a character image was uploaded (not made in FilmLab), including characters whose sheet comes from an uploaded photo. It stands for this statement by the HUMAN: 'Ich habe die Rechte an Bild und Ton dieses Clips und die Einwilligung der gezeigten Personen.' Never set it on your own — show the human the statement and pass true only after they agreed. Without it the backend refuses with rechte_fehlen before anything is charged; get_cost tells you up front (rechte_noetig). | |
| character_ref_urls | No | Ready image URLs that bind the cast's identity. The name says URLs on purpose: lab_animate's `character_refs` takes character sheet IDS, this path takes ADDRESSES, and the two are not interchangeable. Get them from lab_get_character_sheet. Omit to let the backend use the project's world-bible characters. Mutually exclusive with `referenzen`. | |
| reference_video_url | No | URL of the clip whose motion is taken. Use the url/video_url a previous answer gave you (lab_list_assets with asset_type 'video', lab_job_status, lab_review_queue) — never a hand-built address: the bucket is not guessable and a wrong one renders and bills into a 404. Leave it out to run on `referenzen` alone (at least one image reference, no `figur`, no in_sec/out_sec, no depth) — the models that allow it carry referenzen_ohne_video in lab_list_models. |