# Commercial Creator

An agentic pipeline that turns a product description into a finished video
commercial: creative brief → concepts → script → adversarially-reviewed shot
list → styled storyboard keyframes → VO-timed animatic → Seedance 2.0 video
generation (with storyboard-composed multi-shot windows) → QC judging with
corrective regeneration → assembly → music + expressive VO mix → final cut
with themed overlays. A FastAPI server drives it; a single-file web UI and an
MCP connector (for Claude Code and other agents) sit on top.

Built with a **human-in-the-loop by design**: the brief, the shot list and
the animatic are approval gates, and **video generation never spends money
without an explicit approval step** — in the UI and in the MCP tools alike.

## What it produces

30–60s commercials/trailers at draft (480–720p) or final (1080p) tier, with:

- multi-agent planning: intake strategist, creative director (3 concepts,
  beat-scripted VO at ~2.3 words/s), a **shot room** where a cinematographer
  critic and an editor critic attack the draft shot list in parallel before
  the director revises it
- storyboard keyframes (Gemini nano-banana via fal.ai) locked to a style
  recipe, with product reference images for identity
- an **audio-first animatic**: VO takes are generated and measured first,
  the timeline stretches to fit, and the shot plan reconciles the locked
  timeline onto Seedance's 4–15s generation windows (scale / trim / split /
  multi-shot collapse / storyboard-composed windows)
- Seedance 2.0 generation via fal.ai with per-route handling (t2v / i2v /
  reference-composed r2v / last-frame chaining), transient retries, a
  billing fail-fast, and cancellable queue polling
- a 6-dimension vision QC judge that stages corrective regens (never
  auto-spends) and a **self-improvement loop** that distills recurring
  failures into durable prompt rules (per-project lessons + global house
  rules)
- ElevenLabs v3 expressive VO (audio tags per beat kind, per-brand voice
  casting by an agent) + generated music bed, sidechain-ducked under the VO
- Remotion-rendered overlays: theme-palette lower thirds and end cards,
  with a presentation setting (cinematic / balanced / informational) chosen
  at intake or recommended by the brief agent

Typical draft-tier cost: **$10–20 per 30s spot** (Seedance dominates;
LLM/images/VO are cents). See `docs/` for the research that shaped the
pipeline.

## See it in action

Five finished spots produced end-to-end by this pipeline — each was one
product description in, one video out (every brand fictional). Uploaded as
demos on [The Bearded AI Guy](https://www.youtube.com/@bearded-ai-guy):

### 🎬 [PocketBrew — coffee commercial](https://youtu.be/zSaz230ygl4) (45s, ~$17)

The pipeline's first complete spot: theme-styled lower thirds and end card,
ElevenLabs music bed ducked under the VO. Every shot generated per-keyframe
(image-to-video) from the styled storyboard.

<p>
  <a href="https://youtu.be/zSaz230ygl4"><img src="assets/pocketbrew_a.jpg" width="49%" alt="PocketBrew — cabin scene with themed lower third"></a>
  <a href="https://youtu.be/zSaz230ygl4"><img src="assets/pocketbrew_b.jpg" width="49%" alt="PocketBrew — product beat with 3-minute timer"></a>
</p>

### 🛸 [ScoutFly — FOMO drone commercial](https://youtu.be/ddMKuNCeo-Y) (31s, ~$15)

The QC loop on camera: the vision judge failed 4 of 6 draft clips (garbled
text in the blueprint theme), staged corrective regenerations with its
feedback spliced into the prompts, and every fix passed. That failure
pattern then became a permanent theme rule — the self-improvement loop's
origin story.

<p>
  <a href="https://youtu.be/ddMKuNCeo-Y"><img src="assets/scoutfly_a.jpg" width="49%" alt="ScoutFly — blueprint-theme flight path"></a>
  <a href="https://youtu.be/ddMKuNCeo-Y"><img src="assets/scoutfly_b.jpg" width="49%" alt="ScoutFly — the drone with a caller's face on its screen"></a>
</p>

### ⌚ [TIMEPIECE — 60s streaming-series trailer](https://youtu.be/irapcuaeudw) (~$27)

A prestige drama trailer for a show that doesn't exist: custom cinematic
theme, tick-synced cuts, a custom-prompted orchestral score, and a measured
animatic that landed the 60s target exactly.

<p>
  <a href="https://youtu.be/irapcuaeudw"><img src="assets/timepiece_a.jpg" width="49%" alt="TIMEPIECE — cliff beneath three moons"></a>
  <a href="https://youtu.be/irapcuaeudw"><img src="assets/timepiece_b.jpg" width="49%" alt="TIMEPIECE — the proof beat close-up"></a>
</p>

### ☁️ [SKYSILVER — action-comedy trailer](https://youtu.be/neDwwKh56Pw) (31s, ~$14)

The storyboard-composed route at work: two of its montages are SINGLE
Seedance generations following a numbered storyboard grid (multiple hard
cuts per call, scene-detection validated), roughly halving the cost of
per-shot generation. The adversarial shot room made 14 revisions before a
dollar was spent — including a new VO-synced cutaway.

<p>
  <a href="https://youtu.be/neDwwKh56Pw"><img src="assets/skysilver_a.jpg" width="49%" alt="SKYSILVER — brass airship in the cloud sea"></a>
  <a href="https://youtu.be/neDwwKh56Pw"><img src="assets/skysilver_b.jpg" width="49%" alt="SKYSILVER — Captain Silver's wink"></a>
</p>

### ✨ [Wear Your Sparkles — beauty commercial](https://youtu.be/s1I4wbIcFQk) (33s, ~$17)

The hardest photoreal test (macro glitter, skin, eyes): the judge ran 7
corrective regenerations against glitter-physics hallucinations and
distilled two durable lessons about macro realism that now ride every
prompt in the project.

<p>
  <a href="https://youtu.be/s1I4wbIcFQk"><img src="assets/wys_a.jpg" width="49%" alt="Wear Your Sparkles — applicator catching light on the lid"></a>
  <a href="https://youtu.be/s1I4wbIcFQk"><img src="assets/wys_b.jpg" width="49%" alt="Wear Your Sparkles — glitter eye macro with product"></a>
</p>

## Requirements

- **Python 3.12** (pydub/audioop compatibility)
- **ffmpeg + ffprobe** on PATH
- **Node.js 18+** (only for the final overlay render via Remotion; every
  other stage works without it)
- API keys:
  - `FAL_KEY` — [fal.ai](https://fal.ai) (Seedance video + nano-banana images)
  - `ANTHROPIC_API_KEY` — planning/judging agents
  - `ELEVENLABS_API_KEY` — VO + music (optional; the pipeline degrades to
    silent scratch timing without it)

## Setup

```bash
python -m venv .venv
.venv/Scripts/activate            # Windows; source .venv/bin/activate elsewhere
pip install -r requirements.txt

cp .env.example .env              # fill in your keys

uvicorn commercial_server:app --port 8700
# open http://127.0.0.1:8700  (the web UI)
```

Health check: `GET /api/v1/health` reports missing binaries/keys.

> **Google Drive / synced-folder warning:** if this folder lives on a synced
> drive, set `REMOTION_BUILD_DIR` to a local path (e.g. `C:/remotion-build/projects`)
> — `npm install` fails on synced filesystems, and renders stage off-drive
> then copy back. Non-synced installs can set `REMOTION_WORK_MODE=project`.

## The pipeline

```
intake → brief* → script → shots* → refs → storyboard → animatic* → generate† → assemble → audio → variants
         (*human approval gates)                                    (†explicit spend approval)
```

Every stage is a job with live logs (SSE to the UI), and artifacts are
versioned append-only — editing an upstream artifact marks everything
downstream stale rather than deleting it.

## Claude Code / MCP connector

`mcp_server.py` exposes the pipeline as typed MCP tools so an agent can
drive it end-to-end:

```bash
pip install "mcp[cli]"
claude mcp add commercial-creator -- python /absolute/path/to/mcp_server.py
```

Then, in Claude Code: *"Create a 30-second commercial for <product>"* — the
agent walks the stages with `run_step`, honors the approval gates via
`approve_artifact`, and **cannot generate video in one step**:
`stage_video_batch` returns the dollar estimate, and only the separate
`approve_video_jobs` call spends. Point the connector at a non-default
server with `COMMERCIAL_CREATOR_URL`.

## Field notes baked into the code (the expensive lessons)

- **Seedance 2.0 has no seed input** — reproducibility lives in reference
  images + verbatim prompts. Consistency comes from style-recipe keyframes,
  reference binding (`@Image1…`) and single-generation multi-shot windows.
- **The likeness gate**: the provider rejects *detectable photoreal human
  faces* in reference-to-video **reference images and video refs** (facial
  fragments — a macro eye — count). The same images pass as image-to-video
  start frames. Stylized/animated character sheets pass, and carry identity
  across composed multi-shot generations remarkably well. The planner
  routes around all of this automatically via a per-shot `face_prominent`
  flag and a per-project `visual_mode` (photoreal | stylized).
- **Storyboard-composed windows**: a numbered grid of keyframes (≤3×3,
  black borders, number badges) as `@Image1` + timed `Shot N` blocks
  reliably yields one generation with native hard cuts — ~1.2s/shot floor,
  6–7 shots per 10s is the readable ceiling. Cut counts are validated by
  scene detection after download.
- **Output-side "sensitive content" flags are noisy** — one identical retry
  is automatic and usually passes. Input-side likeness rejections are
  deterministic — never retried, rerouted instead.
- **ElevenLabs v3** does not support `previous_text`/`next_text`, and its
  stability tiers map to floats (0.5 = natural). The *voice* is the quality
  lever: conversational-tagged voices respond to `[excited]`-style audio
  tags; narration voices ignore them.
- **Assembly policy**: never concat stream-copied and re-encoded segments;
  uniform per-segment encode trimmed to edit duration, 4-frame crossfades
  on take-chained joins.

**📖 [Full illustrated walkthrough](docs/walkthrough.md)** — installation to
finished commercial, with screenshots of every panel (also published as a
Medium article). `docs/seedance-commercial-workflow-best-practices.md` is
the full research playbook; `docs/seedance-commercial-app-implementation-plan.md`
is the original architecture spec.

## Costs & safety rails

- Per-project **budget cap**: jobs estimate before running; the queue
  refuses to start work that would blow the cap (402) counting in-flight
  estimates.
- All video generation jobs are created `needs_approval` and survive server
  restarts without auto-running.
- The QC judge stages corrective regens with feedback spliced into the
  prompt — approval is still explicit, and after two failed fixes it
  escalates to a human instead of burning more money.

## Repository layout

```
commercial_server.py     FastAPI app (port 8700): REST + SSE + static UI
commercial/              the pipeline package
  agents/                LLM agents (intake, director, breakdown+shot room,
                         reference, judge, continuation) — Claude via tool-forced
                         structured output
  adapters.py            Seedance capability descriptors + fal queue client
  shot_plan.py           timing contract: animatic timeline -> generation rows
  compiler.py            deterministic prompt compiler (inspectable pre-spend)
  storyboard_sheet.py    numbered grid compositor for composed windows
  keyframes.py/animatic.py/assembly.py/audio_vo.py/audio_final.py/music.py
  overlay.py/themes.py   Remotion overlay render + theme palettes
  jobs.py/store.py       approval-gated job queue + single-writer JSON store
  lessons.py             self-improvement: QC failures -> durable prompt rules
static/commercial.html   the whole UI (no build step)
mcp_server.py            MCP connector for Claude Code
remotion_template/       Remotion project (CommercialFinal composition)
```

## License

MIT — see `LICENSE`.