Skip to main content
Glama

ClipKit

The video infra for agents. ClipKit turns a JSON timeline — the open ClipKit Protocol — into rendered video on the GPU. Describe a video as structured data; get an MP4. Built for AI agents and the developers shipping them.

clipkit.dev · Docs · Protocol · Agent quickstarts

https://github.com/user-attachments/assets/ff4b7e9b-2e0a-40f4-a6a1-042e981fc65f

Quick start

npm install -g @clipkit/cli

clipkit new my-video.json                 # scaffold a Source
clipkit render my-video.json -o out.mp4   # render locally (headless Chrome)

…or from code:

import { render } from '@clipkit/renderer';
import { writeFile } from 'node:fs/promises';

const { buffer } = await render({ source });   // source = a ClipKit Protocol document
await writeFile('out.mp4', buffer);

Local rendering uses your installed Google Chrome (for WebCodecs). For GPU-accelerated and pro-format (ProRes / AV1 / transparent) output, render on the hosted service with clipkit render --cloud.

Related MCP server: oh-my-cassette

What's here

A monorepo; each package publishes to npm under the @clipkit scope.

Package

What it is

@clipkit/protocol

the ClipKit Protocol — schema, types, validation

@clipkit/runtime

the engine — WebGPU/WebGL2 compositor + WebCodecs encoder

@clipkit/renderer

render a Source to MP4 locally (headless Chrome)

@clipkit/editor

a configurable, embeddable editor over the schema

@clipkit/editor-core

the editor's UI-free data layer

@clipkit/playback

browser playback engine

@clipkit/patterns

composable motion-graphics units

@clipkit/sfx

procedural sound-effects synthesis

@clipkit/music-analysis

beat / tempo analysis for authoring

@clipkit/speech-to-text

transcription → captions

@clipkit/lint

protocol-aware validation + plain-language summaries

@clipkit/mcp-server

an MCP server so agents can author + render video

@clipkit/cli

the local-first CLI

For AI agents

ClipKit ships an MCP server — point your agent at it and it can build, validate, preview, and render ClipKit videos:

npx -y @clipkit/cli mcp        # stdio MCP server

See the agent quickstarts for Claude, Cursor, and more.

License

Apache-2.0 — except @clipkit/runtime, which is under the Business Source License 1.1 and converts to Apache-2.0 four years after each release. In plain terms: use it, embed it, build your own product on it, and render for your own customers — all free. The one thing you can't do is offer a competing hosted ClipKit-rendering service. See LICENSING.md and TRADEMARK.md.

Available Tools

16 tools
add_elementAdd an element to the current projectAInspect

Append a single element to an existing project — a TWEAK, e.g. dropping in one more caption or shape. By default it is added at the top level; pass parent_id to add it INTO a group (nested). The element is any valid schema element: video, image, text, shape, audio, group, caption, or particles. To create a composition or add several elements at once, build the JSON and use set_project instead. The new element is validated as part of the project as a whole before being added. Call get_schema(element_type) for the exact per-type fields; unrecognized keys are flagged.

Inputs: element (required), parent_id, project_id. Returns: parent_id, element_type, added_element_id, top_level_element_count

create_projectCreate a new Clipkit projectAInspect

Create a new, blank Clipkit project with the given dimensions and duration, and return its project_id. Defaults: 1920×1080, 10 seconds, 30 fps, output_format "mp4". Call this first when starting a new video. Pass an existing project_id to reset that project to blank; omit it to start a fresh project (note the returned id for subsequent tools).

Inputs: width, height, duration, frame_rate, +4 more. Returns: width, height, duration, frame_rate, +2 more

create_promoCompose a designed promo from prebuilt scenes (one fast option)A
Destructive
Inspect

Assemble a designed-looking promo/intro/product/data video from the Clipkit pattern library: give an ordered list of SCENES and the words, and it bakes in the camera, glass, lighting, motion blur, timing, and layout, then returns an editor link. This is a FAST option when a conventional promo structure fits — it is NOT the only way to make a video and NOT a default; for anything specific or original, author the JSON yourself and call set_project (the full creative range). When you do use this, MIX scene types to fit the brief and vary the structure — a video can be a single kinetic headline, three title cards, a showcase montage, or a data explainer; you do NOT need a hero or a cta. Scene types: hero (glass-orb logo reveal: wordmark, tagline?), kinetic (letter-fly headline: text, subtitle?), showcase (a screenshot tilted in 3D: screenshot URL), title (full-frame title card: headline, kicker?, subtitle?), cta (closing card with a glass button: wordmark, tagline?, cta), stats (hero numbers: stats[{label,current,previous?}], title?), bars (bar chart: bars[{label,value,previous?}], title?), ranking (top-N list: items[{label,value}], title?), pie (pie cards: cards[{label,value,total,previous?}], title?). The data scenes (stats/bars/ranking/pie) look best with theme "mux".

Inputs: scenes (required), theme, width, height, +2 more. Returns: width, height, duration, editor_url, +2 more

delete_elementDelete one element by idA
Destructive
Inspect

Delete the element with the given id, anywhere in the tree (including one nested inside a group or its mask) — a tweak to an existing composition. (The project must keep at least one top-level element.)

Inputs: id (required), project_id. Returns: deleted_element_id, top_level_element_count

describe_projectDescribe the current project in plain languageA
Read-only
Inspect

Return a compact, human-readable summary of the current project — dimensions, fps, duration, an element breakdown by type, a per-track timeline (paint order low→high), and render-time warnings. Much cheaper to read than get_project's full JSON; use it to orient yourself or sanity-check structure without dumping the whole source.

Inputs: project_id.

edit_elementTweak one existing element (merge changed fields)AInspect

Change fields on the element with the given id by merging in a partial element — only the keys you include change. The id may be any element ANYWHERE in the tree, including one nested inside a group (or its mask). Pass a whole nested value (e.g. a new keyframe_animations array) to replace that key; set a key to null to remove it. This is for TWEAKING an existing composition. To create a composition or change many elements at once, edit the JSON and call set_project instead. The result is re-validated before being accepted.

Inputs: id (required), patch (required), project_id. Returns: element_id, changed_keys

get_projectGet the current Clipkit project JSONA
Read-only
Inspect

Return the full current Clipkit source as JSON. Use this to inspect the project, pass it to a render pipeline, or compose follow-up edits.

Inputs: project_id.

get_schemaGet the Clipkit JSON Schema (exact fields)A
Read-only
Inspect

Return the authoritative JSON Schema for a Clipkit Source — exact field names, types, and enums, generated from the protocol. Call with no argument for the full Source schema, or with element_type (e.g. "text", "shape", "particles") for just that element's fields (much smaller). Use this when authoring with set_project / add_element so you never guess a field.

Inputs: element_type.

load_projectLoad a shared project back into the sessionA
Destructive
Inspect

Import a previously shared project as the current project, by its share id or its editor URL (e.g. https://clipkit.dev/editor?id=…), returning its project_id. Use this to continue working on a project the user opened in the editor or shared earlier — the round-trip for open_in_editor. Pass an existing project_id to load into that project; omit it to load into a new one.

Inputs: id_or_url (required), project_id. Returns: width, height, duration, project_id, +1 more

open_in_editorCreate a shareable link that opens the current project in the editorAInspect

Validate the current project and create a link that opens it in the Clipkit web editor, where the user can preview and refine it. This shares the PROJECT (nothing is rendered — that's render_video). Use after composing or editing. Returns a URL.

Inputs: project_id. Returns: editor_url, project_id

preview_stillRender one frame of the current project so you can SEE itA
Read-only
Inspect

Render a single frame of the current project to a PNG and return it as an image you can look at. This is how you check your work — in chat there is no other way to see what a composition actually looks like. Use it liberally: after composing, after edits, and at different times to inspect motion. Stills are FREE (credits are only spent by render_video). Pass time (seconds) to choose the frame; defaults to 0.

Inputs: time, project_id. Returns: time, width, height, mime_type

read_docsRead the Clipkit authoring docsA
Read-only
Inspect

Return a canonical Clipkit doc as text. topic "card" = the ~8KB compact authoring card — the recommended context for authoring; "pattern-data-viz" / "pattern-cinematic-ui" / "pattern-ui-screencast" = ~4-5KB archetype pattern cards (proven idioms: count-ups and bar rows; product hero shots with camera rigs; faked app UI with typing/cursor/clicks) — load ONE alongside the card when the brief matches its archetype; "agents" = the full authoring guide (fetch only when the card doesn't cover a need); "protocol" = the formal field spec; "brand" = brand reference. (Same docs offered as MCP resources, exposed as a tool so you can read them directly — resources are not always model-readable.)

Inputs: topic.

render_videoRender the current project to an MP4 in the cloudAInspect

Validate the current Clipkit project and render it to a finished MP4 on Clipkit's servers, returning a downloadable URL. This is the paid path — it consumes render credits and requires CLIPKIT_API_KEY to be configured. Use open_in_editor instead to just open the project in the editor for free. Rendering is synchronous and may take a while for long or high-resolution videos.

Inputs: project_id, resolution. Returns: credits, download_url, duration_seconds

set_projectReplace the entire Clipkit projectA
Destructive
Inspect

Replace the entire project with the given source JSON, returning its project_id. This is the PRIMARY way to build: use it to create a composition or to add/change many elements at once. (To tweak a single element in an existing project, use edit_element / add_element / delete_element instead.) Pass an existing project_id to replace that project; omit it to create a new one (note the returned id). The input is validated against the @clipkit/protocol before being accepted; invalid inputs return an error. Shape: { width, height, duration, frame_rate, output_format, background_color?, fonts?, camera?, lights?, elements:[…] }; every element has a type plus base fields (id, x, y, width, height, time, duration, track, opacity, rotation, animations, keyframe_animations) and type-specific fields. For exact field names + types call get_schema (optionally with an element_type) — the runtime ignores unrecognized keys, and this tool flags any it does not recognize.

Inputs: source (required), project_id. Returns: width, height, duration, project_id, +1 more

transcribe_to_captionsTranscribe speech into a word-timestamped caption elementAInspect

Transcribe an audio or video file into a word-timestamped caption element (the protocol renders these). Captions need real per-word timings, which can only come from actual speech-to-text — this runs Whisper in the server process (no API key, no third-party upload). Provide a url (fetched server-side — use this in chat-mode, where there is no local file) OR a local path. Requires ffmpeg on the host. By default the caption is added to the current project; set add:false to only return it.

Inputs: add, url, path, layer, +3 more. Returns: text, added, word_count, duration_seconds

validate_projectValidate the current project (schema + render-time warnings)A
Read-only
Inspect

Run the @clipkit/protocol validator against the current project AND surface render-time warnings even when the JSON is valid — things that pass the schema but the runtime will silently drop or clip: emoji / non-ASCII text (the runtime font atlas is ASCII-only), elements that run past the composition end, a missing top-level duration. Run it before you share or render the project. For a fuller timeline read-back, use describe_project.

Inputs: project_id. Returns: valid, width, errors, height, +5 more

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.4/5.0

    Scored across 16 tools

    Disambiguation5/5

    Each tool targets a distinct action and object: project CRUD, element tweaks, validation, preview, transcription, promo assembly, editor sharing, and rendering. The descriptions explicitly contrast overlapping tools (set_project vs create_project, get_project vs describe_project), so an agent should never confuse them.

    Naming Consistency5/5

    All 16 tools follow a strict verb_noun pattern with snake_case (create_project, edit_element, render_video). There are no mixed conventions or vague verb-only names, and the verb consistently reflects the operation (get, set, add, edit, delete, validate, preview, render, open, load, transcribe, create).

    Tool Count4/5

    At 16 tools, this is just one over the typical 3-15 sweet spot, but the server's scope is a full video authoring and rendering pipeline, and every tool serves a distinct stage of that pipeline. It feels appropriately scoped rather than bloated.

    Completeness5/5

    The tool surface covers the full lifecycle: project creation and replacement, element CRUD, validation with render warnings, still preview, caption transcription, template-based assembly, editor round-tripping, and paid rendering. There are no obvious dead ends—any operation an agent needs is present.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      B
      quality
      C
      maintenance
      Enables AI coding agents to programmatically assemble vertical videos and manage local CapCut Desktop drafts, including AI background removal, audio mastering to -14 LUFS or Demucs vocal isolation, music auto-ducking, punch zooms, karaoke captions, PiP proof overlays, hook badges, safe-zone linting, proxy previews, and EN→ES cloning/dubbing. It outputs real CapCut draft JSON ready for review and export.
      17
      2
      MIT
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables users to turn raw video clips into finished, beat-synced montage cuts through natural-language chat, automating shot selection, pacing, music matching, subtitles, transitions, picture-in-picture, and rendering. It returns reviewable timeline digests and contact sheets before rendering so edits can be approved in chat.
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to edit video locally through FFmpeg: probing media, validating and previewing JSON edit projects, rendering short-form clips with captions, reframing, transitions and ducked music. Also analyses existing videos and applies or saves reusable format templates, without cloud services or API keys.
      1
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI assistants to automate video editing tasks such as creating, inspecting, and saving projects, adding, splitting, and deleting clips, applying kinetic motion behaviors, and validating render parameters. Supports JSON-RPC 2.0 over stdio and HTTP.
      MIT