Animate Image to Clip
lab_animateAnimate a completed image into a clip with a video model. Pick the model from lab_list_models; check the price with get_cost or let the user confirm a card via lab_propose. Costs credits when called without get_cost.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| seed | No | Seed for a repeatable run. Omit it and the project's stored seed applies (POST /api/lab/projects/{id}/einstellungen), and without that the provider picks one. The seed that actually rendered comes back in lab_job_status as `seed` with `seed_quelle` ('provider' = read from the provider's answer, 'gesetzt' = the one sent, null = the model tells no seed). Models whose endpoint has no seed field do not drop a given seed silently: the run proceeds and the row carries `seed_hinweis: seed_nicht_unterstuetzt` — some models tell their seed but take none, others take one but tell none. | |
| tool | No | Video model id — copy it verbatim from lab_list_models (kind: video). Omit to let the intent router choose. Every model has its own resolutions, duration grid, audio behaviour and price; the catalog is the only place those live, this text deliberately names none. | |
| shots | No | Multi-shot: a sequence of shots rendered as ONE output, holding character, place and voice across the cuts. This is NOT the same as several clips stitched together — those you pay for individually and they do not hold the cast. PRICE: the billed length is the SUM of the shot durations, not duration_seconds. Three 5s shots cost the same as one 15s clip (140 credits at the reference resolution), not the same as one 5s clip (47). The sum must fit the provider's 3-15s window — each shot may be legal on its own and the sum still too long. Only kling_3_pro. Requesting it elsewhere is rejected before quota is spent. | |
| in_sec | No | Cut the reference clip server-side before it reaches the provider: start of the excerpt, in seconds from the clip's beginning. Only together with out_sec, and only with source_video_id. | |
| intent | No | Say WHAT the shot is instead of picking a model — the router chooses by priority and a cost cap. 'draft' and 'ambient' route to the cheapest provider, 'dialog' to the one with the steadiest faces, 'hero' to the most expensive on purpose, 'atmo' for atmosphere. Prefer intent over tool: it also gives you an automatic fallback to the next provider if the first one hard-fails. An explicitly set `tool` always wins over intent. | |
| prompt | Yes | Motion/camera instruction for the animation. For a clip of 4 s or longer, write the action as timed beats, one per line under a '// Action Timing' header: '[0-2s] [CAM 01] <framing> -> [ACT] <action> -> [DIAL] <who>: "<line>" -> [AUDIO] <sound>'. The spans start at 0 and cover duration_seconds; keep the same CAM number while the camera does not cut. lab_prompt_transform (medium video) writes exactly this form. Murnau's SceneFlow lights each beat while the clip plays, and the crew judges it there. | |
| familie | No | Pick the MODEL, not the access route: 'seedance-2.0' or 'seedance-2.5'. The server takes the cheapest route that this account may use and that can do the job (OpenRouter only with the user's consent; references, 1080p for 2.5, or audio switched off go to fal) and falls back to fal if a cheaper route fails — the fallback can cost more than the preflight price. Mutually exclusive with tool and intent. | |
| out_sec | No | End of the excerpt in the reference clip, in seconds. Must lie behind in_sec; both are set together or neither. Only with source_video_id. | |
| get_cost | No | Preflight: return the price in credits and your balance WITHOUT generating, reading the same catalog as lab_list_models. Do this before a batch or when the user wants to see the cost. For a card the user confirms, use lab_propose instead. | |
| metadata | No | Free-form JSON stored alongside the video. Not interpreted by the pipeline. | |
| prop_ids | No | Prop ids from lab_list_props, belonging to this project (an @prop_ tag in the prompt does the same). Animating an image: the descriptions go into the prompt as text, no prop image is sent. From a clip (source_video_id): the front views join the reference images on free slots, as in lab_reference_motion. | |
| audio_url | No | URL of an existing audio track to lay under the clip. Independent of generate_audio, which asks the PROVIDER to synthesise its own. | |
| scene_ref | No | Scene label (e.g. 'SZ 04'). Stored on the row, so clips can be matched back to a shot list and reviewed per scene with lab_review_queue. | |
| shot_type | No | How to cut a multi-shot clip. 'customize' (default) renders exactly the shots you gave. 'intelligent' lets the model decide the cutting — your shot prompts become a suggestion, not a spec. Belongs to `shots`; alone it is rejected. | |
| apply_lens | No | Defaults to TRUE on the backend: the project's lens_state UrPrompt is appended to the prompt. Pass false only to deliberately opt out for this one call. | |
| kamerapfad | No | The camera move as INPUT instead of prose: an object following shared/kamerapfad/schema.json — poses over time (position, look-at or quaternion, metres, Y up) from the Kadrage terrain flight or drawn by hand, or motion phases as lab_kamerafahrt reads them from a clip (its answer carries a ready `kamerapfad`). The backend validates it, derives phases, and then writes the move into the prompt IN THE FORM THIS MODEL UNDERSTANDS: prose at the camera slot for most families, bracketed director commands inside the prompt string for the MiniMax/Hailuo family, which parses them. NO MODEL RECEIVES THE POSES TODAY — the answer says which form went out (kamerapfad.angewandt = als_text or als_regiebefehle), names the family rule and the source it rests on (kamerapfad.sprache), and lists what that form could not carry (kamerapfad.nicht_gesendet, kamerapfad.befehle_weggefallen next to kamerapfad.befehle_gesamt). IF YOUR OWN PROMPT ALREADY CARRIES BRACKETED COMMANDS and the model parses them, the path is NOT sent on top of them — two chains would contradict each other at the same parser; the answer then says angewandt = eigene_regie and names what it found in kamerapfad.eigene_regiebefehle, and your prompt goes out unchanged. A `zusammenfassung` inside the path is display only and is NEVER sent — the move is always rebuilt from the phases; if nothing can be built (phases without tempo/verlauf), the answer says angewandt = nicht_ableitbar with the reason, and nothing of the path is in the prompt. Mutually exclusive with kamerapfad_url. | |
| project_id | Yes | Project id (from lab_list_projects) | |
| referenzen | No | Typed references for the clip path (source_video_id). The position IS the address: the first image is @Image1, the first extra clip is @Video2 (the reference clip itself is @Video1), the first sound is @Audio1. Write @bild1/@video1/@ton1 in the prompt and they are resolved to exactly these positions HERE, before anything is sent — no detour through lab_prompt_transform needed (it resolves them too when it rewrites a prompt). Every image reference is MEASURED before it is sent and scaled up to the model's minimum if it is below it; nothing is ever rejected for being small. | |
| resolution | No | Resolution tier, exactly as listed for the model in lab_list_models. Omit for the model's default. Some models have no resolution field at all — then leave it out. | |
| location_id | No | Location board id (from lab_list_locations). The board's `prompt_prefix_en` is appended to your prompt as 'Location: …', so a clip stays in the same place as the images shot there. A board WITHOUT a prefix anchors nothing and fails silently — check with lab_list_locations. | |
| stil_kit_id | No | Style kit id from lab_list_stil_kits (an @stil_ tag in the prompt does the same). Omit it and the project's default kit applies (lab_stil_kit_standard); its rules and palette go into the prompt as text, and its reference images join only where the call already sends reference images (characters or props) and the model has a free slot after them. The answer says which way it went (stil_kit.weg). | |
| end_image_id | No | id of a completed image to land the clip on (last frame). REQUIRES a start frame (source_image_id or character_refs) — an end frame alone is rejected before quota is spent. Only kling_3_pro. | |
| camera_motion | No | Structured camera move sent to the model itself — only models whose lab_list_models entry lists camera_motions. Prefer this over prose when a keyframe chain must MOVE rather than cross-fade. | |
| ohne_keyframe | No | Skip the keyframe checkpoint on purpose. By default a paid clip only starts from a start image the user has approved (lab_review_asset verdict: accepted) — look at the frame first, then spend on the clip. Set true only when the user explicitly asked to skip the frame check. | |
| ohne_stil_kit | No | true switches the project's default style kit off for this one call. | |
| apply_grundton | No | Defaults to TRUE on the backend: the project's ground tone (the filmmaker's own creative signature, set in the Composer's prompt stage) goes right after the prompt. Pass false only to deliberately opt out for this one call. | |
| character_refs | No | Character ids (positive integer strings) whose reference image locks the cast's identity for this clip. On most providers only the FIRST character's image is used, as the start frame — with two characters in frame the second is unbound. On kling_3_pro each character additionally becomes its own reference element (frontal view plus up to 3 further angles from its sheet), so all of them are bound. The response reports `elements_bound` and `characters_without_reference`. | |
| generate_audio | No | Ask for the model's native audio track. Only meaningful where lab_list_models says audio: schaltbar — models with audio: immer always have it, audio: nie never. | |
| kamerapfad_url | No | Address of a kamerapfad JSON instead of the object — the url of a file in THIS project (lab_list_assets), e.g. the kamerapfad JSON that Kadrage writes next to its flight video. Fetched by the backend under the same ownership rule as every reference: a file of another project is a named rejection before anything is charged. Mutually exclusive with kamerapfad. | |
| source_image_id | No | id of a completed image (from lab_generate/lab_job_status) to animate — positive integer as string. Required unless get_cost is true. Imported Midjourney images work directly: use an image_id from lab_umzug. | |
| source_video_id | No | Animate FROM A CLIP instead of a still: the id of a generated video (from lab_animate/lab_job_status) or the asset key of an uploaded one (from lab_list_assets, asset_type 'video'). The run then goes to the reference-motion path — your clip leads, the model restyles it. Mutually exclusive with source_image_id, and only for models that take a video reference (lab_list_models says v2v); The picture-side fields (end frame, shots, audio, location) do not travel this path and are rejected rather than dropped; character references exist here under a different name (ready image URLs instead of sheet ids) and are rejected too, with that reason. An intent IS resolved before the run and travels as the chosen model. | |
| character_states | No | Which STATE each character is in for this clip, as { characterId: stateKey } — e.g. { "12": "wet" }. A state is its own asset with its own full descriptor (clean / soaked / bloodied are three assets, not one with a note), and it is the ONLY way that descriptor reaches a generation: the reference image always carries the BASE state, so without this field you get dry Cal in a scene where he just came out of the Thames — and the clip looks correct. Get the keys from lab_get_character_sheet. Every id here must also appear in character_refs (who is not in the frame is not described), and an unknown key is rejected before any quota is spent. | |
| character_voices | No | Bind a voice per character: character id -> fal voice id. Voice ids come from fal-ai/kling-video/create-voice — FilmLab keeps no voice registry of its own, the id is passed through unverified. Every character named here must also be in character_refs AND must have a generated reference image (no image → no element → nothing to bind the voice to; rejected before quota is spent). Only kling_3_pro. | |
| dreamer_signatur | No | Proof from the Dreamer (lab_prompt_transform, or the dreamer_fassung of a previous call) that THIS prompt went through it. Without a valid one the tool does not generate: it runs the Dreamer and returns its version first. | |
| duration_seconds | No | Clip length in seconds. Must lie inside the model's duration grid from lab_list_models; inside the grid the provider may snap to its nearest step. IGNORED when `shots` is given — then the rendered length is the sum of the shot durations. | |
| nur_eigene_daten | No | Only for a model that sends inputs to China or a US recipient without a documented basis, when the step carries images, clips, voices or faces. Means: this material shows or contains no real person other than the human using this account. Set it ONLY after the human has said exactly that for THIS step; never assume it, never reuse it for the next step. Not needed when the human has declared this material in the FilmLab app: the backend reads that declaration, also for material derived from it; only the app can store one. An LK reference, a licensed likeness or another person's agreement does not replace it. Without it the backend refuses with drittland_dritte_gesperrt before any charge; then use a non-gated model or text only. | |
| abdruck_startframe | No | Generate frame one WITH the character's trained Figurenabdruck (LoRA) instead of taking the flat studio portrait from the character sheet — the clip then starts IN the scene rather than in a photo studio. Opt-in: it costs one extra image (3 credits). Needs EXACTLY ONE character_ref that has a finished Abdruck, and is rejected together with source_image_id (that one PICKS frame one, this one MAKES it). Refused before any quota is spent. | |
| veraltete_figur_ok | No | Only with source_video_id (the clip path); animating an image rejects it. Exception for a reference clip made with an OLDER version of a character: such a clip is refused (figur_veraltet, nothing charged). Set it to true ONLY after the human has seen that warning and said yes to going on with the old version — ask first, name the price, wait for the yes; never set it on your own. Default off. The alternative is to re-create the clip with the current version of the figure. |