Cleya

Status: v0.6.0, actively maintained. 834 tests across 75 files, a documented scene grammar with a JSON Schema beside it, and a contract-tested starter library — a change that breaks a starter breaks the build. Issues and pull requests welcome.

What it is

67 tools on three surfaces — a CLI, an MCP server, and a TypeScript library — covering the whole path from "I have a brief" to "here is a measured, on-brand MP4":

• Author. Describe an animation as declarative JSON (the scene DSL) instead of hand-writing HTML and timing code. Layers, entrances, idle motion, keyframe tracks, camera moves, tickers, data binding, multi-format output.

• Render. Headless Chrome samples window.CLEYA.seekTo(t) once per frame, so every frame is a pure function of time — deterministic, seekable, resolution-independent — then FFmpeg encodes.

• Measure. Perception instruments read the rendered pixels back: WCAG contrast per text box, optical alignment, dominant-colour palette, brand conformance by ΔE. The agent that made the creative can see whether the creative is actually any good.

That last step is the point. Generation is commodity; the loop is made → measured → corrected, and it runs without a human in it.

Related MCP server: flash-cast-mcp

The loop, end to end

# 1. start from a guaranteed-valid skeleton
cleya starters --name promo-hero --copy-to ./promo

# 2. render it with your brand tokens
cleya render --scene promo/promo-hero.scene.json --brand brand.json --output promo.mp4

# 3. measure what came out
cleya contrast    --input promo.mp4 --time 3.2                    # WCAG per OCR'd text box
cleya align       --input promo.mp4 --time 3.2                    # near-miss edges, broken rhythm
cleya brand_check --input promo.mp4 --time 3.2 --brand brand.json # 0-100 ΔE conformance

# 4. correct deterministically — refinement as code, not deliberation
cleya edit --scene promo/promo-hero.scene.json --op '{"op":"center","layer":"headline"}' --verify

# 5. promote a good result into a reusable starter
cleya harvest --scene promo/promo-hero.scene.json --brand brand.json \
  --name book-flip-promo --when "Book-brand promo with flipping titles"

Or collapse steps 1–3 into one call:

cleya draft --starter promo-hero --brand brand.json \
  --copy '{"headline":"Ship it {accent:Friday}","tagline":"in one command","stat":42}' \
  --output draft.mp4
# → rendered + contrast-checked + brand-scored, ~15s
# (layer ids and knobs per starter: cleya starters --name promo-hero)

Quick start

Cleya is not published to npm. Install from source:

git clone https://github.com/severment/cleya.git
cd cleya && npm install && npm run build && npm link
cleya doctor          # check FFmpeg, ffprobe, Chrome, Whisper
cleya recipes         # goal → tool-sequence discovery

As an MCP server

{
  "mcpServers": {
    "cleya": { "command": "cleya-mcp" }
  }
}

Every tool is registered as cleya_<tool> with a typed schema. Point an agent at cleya_capabilities first — one call returns the whole machine-readable manifest (options, examples, categories, cost hints), which is how an agent learns the toolkit without guessing.

As a library

import { render, contrast, brandCheck, resolveConfig } from "cleya";

const config = resolveConfig();
const out = await render({ scene: "promo.json", brand: "brand.json", output: "promo.mp4" }, config);
const wcag = await contrast({ input: "promo.mp4", time: 3.2 }, config);

Every tool takes (params, config) and returns Promise<ToolResult<T>>.

Authoring — the scene DSL

A scene is data, not code. The compiler turns it into the deterministic seekTo runtime that the capture backends drive:

{
  "size": [1920, 1080], "duration": 10, "bg": "$palette.bg",
  "layers": [
    { "id": "h1", "type": "text", "text": "MAKE IT MOVE", "at": { "x": 118, "y": 208 },
      "style": { "size": 104, "weight": 900, "color": "$palette.ink", "uppercase": true },
      "in": { "at": 0.45, "dur": 0.75, "from": "rise", "ease": "outCubic" } },
    { "id": "cta", "type": "text", "text": "Get started", "anchor": "bottom-left", "safe": true,
      "style": { "bg": "$palette.accent", "pad": [18, 34], "radius": 999 },
      "in": { "at": 2.1, "dur": 0.5, "from": "pop" },
      "idle": { "kind": "breathe", "amp": 0.02 } }
  ]
}

• --check validates without rendering (~0.05s) and warns on dead air.

• --at <t> renders one frame; --preview writes a labelled contact sheet — this is how an agent "watches" a video without watching it.

• --format og,square,story emits every size from one spec, with per-layer overrides.

• --brand brand.json resolves $brand.* / $palette.* tokens and merges fonts.

• --data / --data-each bind typed values and batch-render one spec into N named outputs.

• cleya scrub opens a film in a self-contained scrub page — timeline with hover-scrub, frame-stepping, loop — with zero rendering, because it plays through the same seekTo contract. With --brand, it also carries a live palette bar: re-tint any brand role and the film updates instantly, then download the edited brand.json to render with. Works for raw --html films too, as long as their colours are authored as var(--cleya-<role>, <hex>).

type: "video" layers put real footage on the same timeline — the runtime seeks currentTime per frame rather than playing, so a cut is still a pure function of scene time.

Full grammar: docs/scene-spec.md · schema: docs/scene.schema.json · craft guidance for ambitious work: docs/craft.md

When the DSL genuinely isn't enough, render --html --runtime gsap is the escape hatch: it injects GSAP with every plugin registered plus window.cleya helpers, and you author a paused timeline that the same seekTo bridge drives.

Skills

skills/ ships agent-readable craft guidance, one directory per lens — read before authoring, not after:

Perception instruments

The part nothing else in this category ships. All of these read rendered pixels, not source specs.

They work on any video, not just one Cleya rendered. contrast, align, palette, brand_check, review and pacing take --input <file> and measure whatever is in it. Point them at output from another renderer, at footage from an editor, at a file someone sent you. The measuring half of this toolkit is not coupled to the authoring half, so you can adopt it without moving anything you already have. (scrub is the exception — it needs a seekTo film to drive.)

harvest and edit --verify are instrument-gated: alignment defects block, contrast findings warn. A defective scene cannot silently enter your starter library.

The rest of the toolkit

Run cleya --help for all 67, or cleya capabilities for the machine-readable manifest.

• Author — render, screenshot, card, title, caption, still, tts, capture (scripted real-app screen recording with synthetic cursor, auth state seeding, dev-overlay hiding)

• Pipeline — draft, compose (multi-scene assembly with transitions, music bed, ducking, beat-snap, per-scene VO and content-hashed render cache), social_clip, autosub

• Edit — trim, concat, speed, reformat, audio, overlay, pip, color, lut, effects, shake, upscale, gif, kenburns, layout, multiclip, sync, bg_remove

• Scout — info, scenes, motion, loudness, beat_detect, cut_silence, thumbnails (saliency-ranked), transcribe, detect_text, image_similarity

• Assets — cutout (Vision foreground segmentation to transparency), smart_crop, brand_extract, segment, lut_extract

• Export — export_nle (FCPXML), translate_srt

Requirements

Cleya runs on macOS, Linux and Windows. 53 of the 67 tools are platform-independent, including the whole core loop: render, draft, starters, edit, harvest, scrub, compose, brand_check, palette, claims and diff. Nine are macOS-only (the Vision-backed row above), and five degrade rather than fail: review keeps its FFmpeg checks, smart_crop falls back to a centre crop, thumbnails loses saliency ranking, reformat loses crop:auto, autosub loses the animated style.

cleya doctor checks all of it at once.

Result contract

{ success: true, output: "/abs/path", tool, cleya_version, duration_ms, ...toolSpecificData }
{ success: false, error: "Human-readable message", code: "ERROR_CODE", suggestion?: "What to do" }

Codes are stable and actionable — VALIDATION_ERROR, FFMPEG_ERROR, WHISPER_NOT_FOUND, DRAWTEXT_NOT_FOUND, OUTPUT_EXISTS, NO_CANDIDATES, and more. Agents branch on the code; humans read the message.

Configuration

const config = resolveConfig({
  ffmpeg_path: "/opt/homebrew/bin/ffmpeg",
  temp_dir: "/tmp/cleya",
  ffmpeg_timeout: 600_000,
});

Render tuning lives in the environment: CLEYA_RENDER_WORKERS (parallel frame capture), CLEYA_NO_GPU=1 (software raster), CLEYA_RENDER_ENGINE=chrome|webkit, CLEYA_STARTERS_DIR.

Development

npm run build        # tsc
npm test             # vitest — 834 tests across 75 files
npm run smoke        # end-to-end render smoke test

Starter scenes are contract-tested: every shipped starter must compile with zero errors and zero warnings against the neutral default brand, so a change that breaks a starter breaks the build.

License

Apache 2.0. See NOTICE for third-party attribution — in particular GSAP, which is used by the raw-HTML runtime, is licensed under GreenSock's standard "no charge" license (not Apache-2.0) and is installed from npm rather than redistributed here.