<p align="center">
  <img src="assets/cover.png" alt="Cleya — creative tools that check their own work" width="100%">
</p>

<h1 align="center">Cleya</h1>

<p align="center"><em>A creative-generation and video toolkit for AI agents. Works with any LLM to make beautiful creative, from statics to 2D animation, then <strong>checks its own work</strong> before handing it over.</em></p>

<p align="center">
  <a href="https://github.com/severment/cleya/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/severment/cleya/ci.yml?branch=main&style=flat-square&label=tests&labelColor=14231F&color=17958A"></a>
  <img alt="license" src="https://img.shields.io/badge/license-Apache--2.0-17958A?style=flat-square&labelColor=14231F">
  <img alt="node" src="https://img.shields.io/badge/node-%E2%89%A518-17958A?style=flat-square&labelColor=14231F">
  <img alt="MCP" src="https://img.shields.io/badge/MCP-ready-2DD4BF?style=flat-square&labelColor=14231F">
  <img alt="surfaces" src="https://img.shields.io/badge/CLI_·_MCP_·_library-2DD4BF?style=flat-square&labelColor=14231F">
</p>

---

**Status: v0.6.0, actively maintained.** 834 tests across 75 files, a documented scene grammar
with a JSON Schema beside it, and a contract-tested starter library — a change that breaks a
starter breaks the build. Issues and pull requests welcome.

---

## What it is

67 tools on three surfaces — a CLI, an MCP server, and a TypeScript library — covering the
whole path from "I have a brief" to "here is a measured, on-brand MP4":

- **Author.** Describe an animation as declarative JSON (the scene DSL) instead of hand-writing
  HTML and timing code. Layers, entrances, idle motion, keyframe tracks, camera moves, tickers,
  data binding, multi-format output.
- **Render.** Headless Chrome samples `window.CLEYA.seekTo(t)` once per frame, so every frame is
  a pure function of time — deterministic, seekable, resolution-independent — then FFmpeg encodes.
- **Measure.** Perception instruments read the *rendered pixels* back: WCAG contrast per text box,
  optical alignment, dominant-colour palette, brand conformance by ΔE. The agent that made the
  creative can see whether the creative is actually any good.

That last step is the point. Generation is commodity; the loop is **made → measured → corrected**,
and it runs without a human in it.

## The loop, end to end

```bash
# 1. start from a guaranteed-valid skeleton
cleya starters --name promo-hero --copy-to ./promo

# 2. render it with your brand tokens
cleya render --scene promo/promo-hero.scene.json --brand brand.json --output promo.mp4

# 3. measure what came out
cleya contrast    --input promo.mp4 --time 3.2                    # WCAG per OCR'd text box
cleya align       --input promo.mp4 --time 3.2                    # near-miss edges, broken rhythm
cleya brand_check --input promo.mp4 --time 3.2 --brand brand.json # 0-100 ΔE conformance

# 4. correct deterministically — refinement as code, not deliberation
cleya edit --scene promo/promo-hero.scene.json --op '{"op":"center","layer":"headline"}' --verify

# 5. promote a good result into a reusable starter
cleya harvest --scene promo/promo-hero.scene.json --brand brand.json \
  --name book-flip-promo --when "Book-brand promo with flipping titles"
```

Or collapse steps 1–3 into one call:

```bash
cleya draft --starter promo-hero --brand brand.json \
  --copy '{"headline":"Ship it {accent:Friday}","tagline":"in one command","stat":42}' \
  --output draft.mp4
# → rendered + contrast-checked + brand-scored, ~15s
# (layer ids and knobs per starter: cleya starters --name promo-hero)
```

## Quick start

Cleya is not published to npm. Install from source:

```bash
git clone https://github.com/severment/cleya.git
cd cleya && npm install && npm run build && npm link
cleya doctor          # check FFmpeg, ffprobe, Chrome, Whisper
cleya recipes         # goal → tool-sequence discovery
```

### As an MCP server

```json
{
  "mcpServers": {
    "cleya": { "command": "cleya-mcp" }
  }
}
```

Every tool is registered as `cleya_<tool>` with a typed schema. Point an agent at
`cleya_capabilities` first — one call returns the whole machine-readable manifest (options,
examples, categories, cost hints), which is how an agent learns the toolkit without guessing.

### As a library

```typescript
import { render, contrast, brandCheck, resolveConfig } from "cleya";

const config = resolveConfig();
const out = await render({ scene: "promo.json", brand: "brand.json", output: "promo.mp4" }, config);
const wcag = await contrast({ input: "promo.mp4", time: 3.2 }, config);
```

Every tool takes `(params, config)` and returns `Promise<ToolResult<T>>`.

## Authoring — the scene DSL

A scene is data, not code. The compiler turns it into the deterministic `seekTo` runtime that the
capture backends drive:

```jsonc
{
  "size": [1920, 1080], "duration": 10, "bg": "$palette.bg",
  "layers": [
    { "id": "h1", "type": "text", "text": "MAKE IT MOVE", "at": { "x": 118, "y": 208 },
      "style": { "size": 104, "weight": 900, "color": "$palette.ink", "uppercase": true },
      "in": { "at": 0.45, "dur": 0.75, "from": "rise", "ease": "outCubic" } },
    { "id": "cta", "type": "text", "text": "Get started", "anchor": "bottom-left", "safe": true,
      "style": { "bg": "$palette.accent", "pad": [18, 34], "radius": 999 },
      "in": { "at": 2.1, "dur": 0.5, "from": "pop" },
      "idle": { "kind": "breathe", "amp": 0.02 } }
  ]
}
```

- `--check` validates without rendering (~0.05s) and warns on dead air.
- `--at <t>` renders one frame; `--preview` writes a labelled contact sheet — this is how an agent
  "watches" a video without watching it.
- `--format og,square,story` emits every size from one spec, with per-layer overrides.
- `--brand brand.json` resolves `$brand.*` / `$palette.*` tokens and merges fonts.
- `--data` / `--data-each` bind typed values and batch-render one spec into N named outputs.
- `cleya scrub` opens a film in a self-contained scrub page — timeline with hover-scrub,
  frame-stepping, loop — with zero rendering, because it plays through the same `seekTo` contract.
  With `--brand`, it also carries a **live palette bar**: re-tint any brand role and the film
  updates instantly, then download the edited `brand.json` to render with. Works for raw `--html`
  films too, as long as their colours are authored as `var(--cleya-<role>, <hex>)`.

`type: "video"` layers put real footage on the same timeline — the runtime seeks
`currentTime` per frame rather than playing, so a cut is still a pure function of scene time.

Full grammar: [`docs/scene-spec.md`](docs/scene-spec.md) · schema: [`docs/scene.schema.json`](docs/scene.schema.json) ·
craft guidance for ambitious work: [`docs/craft.md`](docs/craft.md)

When the DSL genuinely isn't enough, `render --html --runtime gsap` is the escape hatch: it
injects GSAP with every plugin registered plus `window.cleya` helpers, and you author a paused
timeline that the same `seekTo` bridge drives.

## Skills

`skills/` ships agent-readable craft guidance, one directory per lens — read before authoring,
not after:

| Skill | For |
|---|---|
| **edit-craft** | The grammar of cuts: which cut, when to cut, ad pacing, and how each maps onto layer windows |
| **marketing-producer** | Campaign work — offer, proof, CTA |
| **social-editor** | Vertical, sound-off, first-three-seconds |
| **documentary-editor** | Long-form, interview-led |
| **podcast-producer** | Multi-speaker audio-first |

## Perception instruments

The part nothing else in this category ships. All of these read rendered pixels, not source specs.

**They work on any video, not just one Cleya rendered.** `contrast`, `align`, `palette`,
`brand_check`, `review` and `pacing` take `--input <file>` and measure whatever is in it. Point
them at output from another renderer, at footage from an editor, at a file someone sent you. The
measuring half of this toolkit is not coupled to the authoring half, so you can adopt it without
moving anything you already have. (`scrub` is the exception — it needs a `seekTo` film to drive.)

| Tool | What it measures |
|---|---|
| **contrast** | WCAG ratio per OCR'd text box — strokes split from ground by Otsu, so it measures the text, not the box |
| **align** | Infers the frame's alignment systems, then flags only a lone element that almost joins one — two deliberate systems a few px apart are not a defect. Plus broken vertical rhythm |
| **palette** | Coverage-weighted dominant colours (ΔE-merged) + luminance stats |
| **brand_check** | 0–100 brand conformance against a `BrandProfile` by ΔE, neutrals passing; optional scene font check |
| **review** | Render QA — black/empty/frozen frames, text collisions, unsafe margins, tiny text, audio dead air; `--platform tiktok\|reels\|shorts` adds UI-occlusion zones |
| **claims** | Lints copy against a claims policy — banned phrases, required disclaimer, line length, reading grade |
| **pacing** | The film's **rhythm**: holds, beats, segment lengths, end-card dwell, plus an activity strip. The only instrument that reads the film as a film rather than a frame, and the only way to check "the end card holds for three seconds" without watching it |
| **diff** | **Where** two frames differ (bounding box), by how much (per-channel max), PSNR, and a difference image amplified to the change it actually contains |

`harvest` and `edit --verify` are instrument-gated: alignment defects block, contrast findings warn.
A defective scene cannot silently enter your starter library.

## The rest of the toolkit

Run `cleya --help` for all 67, or `cleya capabilities` for the machine-readable manifest.

- **Author** — `render`, `screenshot`, `card`, `title`, `caption`, `still`, `tts`, `capture`
  (scripted real-app screen recording with synthetic cursor, auth state seeding, dev-overlay hiding)
- **Pipeline** — `draft`, `compose` (multi-scene assembly with transitions, music bed, ducking,
  beat-snap, per-scene VO and content-hashed render cache), `social_clip`, `autosub`
- **Edit** — `trim`, `concat`, `speed`, `reformat`, `audio`, `overlay`, `pip`, `color`, `lut`,
  `effects`, `shake`, `upscale`, `gif`, `kenburns`, `layout`, `multiclip`, `sync`, `bg_remove`
- **Scout** — `info`, `scenes`, `motion`, `loudness`, `beat_detect`, `cut_silence`, `thumbnails`
  (saliency-ranked), `transcribe`, `detect_text`, `image_similarity`
- **Assets** — `cutout` (Vision foreground segmentation to transparency), `smart_crop`,
  `brand_extract`, `segment`, `lut_extract`
- **Export** — `export_nle` (FCPXML), `translate_srt`

## Requirements

| Requirement | Needed for | Notes |
|---|---|---|
| Node ≥ 18 | everything | |
| FFmpeg + ffprobe | everything | `brew install ffmpeg`. Invoked as a separate process; set `ffmpeg_path` to point elsewhere |
| Chrome / Chromium | `render`, `screenshot`, `capture` | Auto-detected; override with `CLEYA_CHROME`. Flags via `CLEYA_CHROME_FLAGS` |
| FFmpeg with libfreetype | `card`, `caption`, `autosub` | Default Homebrew formula lacks it: `brew tap homebrew-ffmpeg/ffmpeg && brew install homebrew-ffmpeg/ffmpeg/ffmpeg` |
| whisper.cpp | `transcribe`, `autosub`, `clean_speech`, `animated_captions` | Returns `WHISPER_NOT_FOUND` if absent |
| macOS + Xcode CLT | `contrast`, `align`, `detect_text`, `caption_extract`, `cutout`, `segment`, `bg_remove`, `animated_captions`, `image_similarity` | Vision-backed, so these nine need a Mac. Native helpers compile on first use and cache |
| `OPENAI_API_KEY` | `tts` only | Opt-in cloud; nothing else calls out |

**Cleya runs on macOS, Linux and Windows.** 53 of the 67 tools are platform-independent,
including the whole core loop: `render`, `draft`, `starters`, `edit`, `harvest`, `scrub`,
`compose`, `brand_check`, `palette`, `claims` and `diff`. Nine are macOS-only (the Vision-backed
row above), and five degrade rather than fail: `review` keeps its FFmpeg checks, `smart_crop`
falls back to a centre crop, `thumbnails` loses saliency ranking, `reformat` loses `crop:auto`,
`autosub` loses the animated style.

`cleya doctor` checks all of it at once.

## Result contract

```typescript
{ success: true, output: "/abs/path", tool, cleya_version, duration_ms, ...toolSpecificData }
{ success: false, error: "Human-readable message", code: "ERROR_CODE", suggestion?: "What to do" }
```

Codes are stable and actionable — `VALIDATION_ERROR`, `FFMPEG_ERROR`, `WHISPER_NOT_FOUND`,
`DRAWTEXT_NOT_FOUND`, `OUTPUT_EXISTS`, `NO_CANDIDATES`, and more. Agents branch on the code;
humans read the message.

## Configuration

```typescript
const config = resolveConfig({
  ffmpeg_path: "/opt/homebrew/bin/ffmpeg",
  temp_dir: "/tmp/cleya",
  ffmpeg_timeout: 600_000,
});
```

| Key | Default | Purpose |
|---|---|---|
| `ffmpeg_path` / `ffprobe_path` | `"ffmpeg"` / `"ffprobe"` | Binary locations |
| `whisper_path` | `"whisper"` | whisper.cpp binary |
| `temp_dir` | `"/tmp/cleya"` | Working files |
| `retention_hours` | `24` | Temp cleanup |
| `output_dir` | `"."` | Default output directory |
| `ffmpeg_timeout` / `whisper_timeout` | `300000` / `600000` | Execution timeouts (ms) |
| `platform_presets` | see `presets.ts` | Per-platform size/codec/duration |

Render tuning lives in the environment: `CLEYA_RENDER_WORKERS` (parallel frame capture),
`CLEYA_NO_GPU=1` (software raster), `CLEYA_RENDER_ENGINE=chrome|webkit`, `CLEYA_STARTERS_DIR`.

## Development

```bash
npm run build        # tsc
npm test             # vitest — 834 tests across 75 files
npm run smoke        # end-to-end render smoke test
```

Starter scenes are contract-tested: every shipped starter must compile with **zero errors and zero
warnings** against the neutral default brand, so a change that breaks a starter breaks the build.

## License

[Apache 2.0](LICENSE). See [NOTICE](NOTICE) for third-party attribution — in particular GSAP, which
is used by the raw-HTML runtime, is licensed under GreenSock's standard "no charge" license (not
Apache-2.0) and is installed from npm rather than redistributed here.