aidemo
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| OPENAI_API_KEY | No | API key for OpenAI TTS (required if AIDEMO_TTS_PROVIDER is not 'local') | |
| OPENAI_BASE_URL | No | Optional base URL for OpenAI-compatible TTS endpoint (e.g., a local server). | |
| AIDEMO_TTS_PROVIDER | No | TTS provider; set to 'local' to use in-process voice (no API key needed). Default is 'openai' if key is set, else 'local'. |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
aidemo renders narrated, captioned product-demo videos from a storyboard.json. Start by calling get_authoring_guide and follow it. Validate storyboard edits with validate_storyboard (cheap) before rendering. Pipeline operations (probe, record, render, voice, captions, compose, gif) return a jobId immediately — poll job_status for progress, per-scene counters, results, and failure artifacts; only one job runs at a time. Always pass absolute demo directories. Hit an engine bug, surprise, or workaround this session? Call the feedback tool (title + body — environment/log context is auto-attached) before you finish; skip if nothing came up. A failed job_status result also carries a feedbackHint pointing back here.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| resources | {
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| get_authoring_guideA | The canonical guide to authoring demos: storyboard schema, action vocabulary, demo-director principles, ChatGPT-app recording facts. Call this FIRST before authoring or editing a storyboard. Pass |
| get_storyboard_schemaA | JSON Schema for storyboard.json, generated from the engine's own zod schema — the exact contract the engine validates against. |
| validate_storyboardA | Validate a storyboard against the engine schema without running anything. Pass exactly one of: dir (uses generated/storyboard.json), path (a storyboard file), or json (storyboard JSON as a string). relaxed makes narration optional (probe semantics). |
| lint_storyboardA | Browser-free preflight over a storyboard: a per-scene pacing forecast (which scenes compose will mostly freeze-hold or cut because the narration and the recorded action don't match), selector/wait pitfalls (type+Enter with no wait, :text-is on nested labels, focus without a zoom block, anchored waitForChange regexes), and no-op keys. Run it after every storyboard edit, before probe/render. Pass exactly one of dir / path / json. lang lints a narrations[lang] translation at that language's speaking rate. |
| init_demoA | Create demos// with a starter brief + storyboard. dir is the repo to scaffold into (absolute path recommended; default: server cwd). With fromUrl the page is inspected first (no LLM): its headings become scenes and its unique selectors become the beats + a |
| doctorA | Check prereqs: node, ffmpeg, Chrome (or the AIDEMO_CHROME_CHANNEL / AIDEMO_CHROME_EXECUTABLE browser in use), TTS/STT endpoint (flags LLM-only servers like Ollama), API key, playwright + its ffmpeg helper (records the take; installed on the first take unless AIDEMO_NO_AUTO_INSTALL=1), installed skill. |
| embedA | Ready-to-paste embed snippets (markdown GIF, markdown still, HTML ) for a demo, using stable raw.githubusercontent URLs on the 'demo-media' branch. Pure string generation — no network, no render. Owner/repo come from the repo's origin remote; the demo name from the directory basename. Pair with the demo-publish workflow so CI keeps the URLs fresh. See docs/EMBEDS.md. |
| feedbackA | File a bug, surprise, or workaround you hit this session as a GitHub issue on the aidemo engine repo (github.com/tandryukha/aidemo) — the in-band equivalent of |
| job_statusB | Status, stage, per-scene progress (scenesTotal/scenesDone/currentScene — populated for probe/record/render/voice/captions/compose), log tail, and final result or error (with failure-artifact paths and, on a genuine failure, a feedbackHint pointing at the feedback tool) of a pipeline job. |
| job_listA | All jobs this server session, newest last. |
| job_cancelA | Best-effort cancel (checked between actions/scenes/stages). A mid-record cancel salvages the partial timeline + footage. The job settles asynchronously — poll job_status for the final state. |
| probeA | Record-only dry run to verify selectors/timing (narration optional). With updateGolden, writes golden/probe.json (the regression baseline); with golden, deep-compares against it and returns match + field-level diffs. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| import_traceA | Turn a Playwright trace.zip (context.tracing / |
| recordA | Drive the storyboard in Chrome and record raw video + timeline.json. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| renderA | Full pipeline: voice → record → captions → compose (the CLI |
| voiceA | Generate per-scene TTS narration (skips unchanged scenes). Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| captionsA | Transcribe narration.mp3 to captions with word timing. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| composeA | Trim, sync, mux and caption into output/final-demo.mp4 (+ final-demo.srt/.vtt timed to it). Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| qaA | Post-render checks on the FINISHED MP4 — blank opening, blank poster, master loudness, music-bed level, per-scene hold shares + compose warnings, caption readability, frame/chapter hygiene. Run it after compose/render: it sees what a viewer sees, which neither lint (pre-take) nor report.json (structural) can. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| gifB | Convert output/final-demo.mp4 to a README-ready GIF. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| stillsA | Extract named stills (screenshot mode) from the recorded take into output/stills/ — one PNG per |
| framesA | Dump evenly spaced PNG frames from the final video (or the raw take) into output/frames/ for review — look at them instead of hand-running |
| walkthroughA | Export output/walkthrough/ from the final video: index.html (one card per scene — payoff frame, title, narration, jump-to-time; ← → keyboard nav), guide.md (README/SOP-ready), per-scene PNGs, SRT/VTT and a JSON manifest. Needs only the rendered video + report.json (no key, no browser). Returns a jobId immediately — poll job_status. Rejects if another job is running. |
| inspectA | Open a URL in the recording profile (logged-in state included) and list every visible interactive element with UNIQUE selectors ranked data-testid → id → aria-label → role/text → name/placeholder → class → path, plus headings and iframes. Use it BEFORE writing targets — no selector guessing, no wasted probe. Writes logs/inspect-.json and a screenshot in the demo dir. Returns a jobId immediately — poll job_status. Rejects if another job is running. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| authoring-guide | Canonical agent-neutral demo-authoring guide |
| storyboard-schema | JSON Schema generated from the engine's zod schema |
TDQS
Scored across 24 tools
Most tools occupy clearly distinct slots: pipeline stages (voice, record, captions, compose, render), job control (job_status/job_list/job_cancel), and authoring aids (guide, schema, validate, lint). The riskiest boundaries are probe vs record vs render (dry run vs raw take vs full pipeline) and validate_storyboard vs lint_storyboard, but the descriptions explicitly draw those lines. Mild residual overlap between compose and render, and stills vs frames, is handled by clear scoping text.
Three conventions coexist: bare pipeline stage names (record, voice, compose, qa, gif, stills, frames, walkthrough), a job_* prefix (job_status, job_list, job_cancel), and verb_noun authoring helpers (get_authoring_guide, init_demo, validate_storyboard, import_trace). Every name is readable and the grouping is quasi-intentional, but it is not a single predictable pattern, so an agent can't infer a name from an action.
At 24 tools the surface is on the heavy side, but each tool maps to a genuinely distinct CLI stage or analysis step rather than being redundant. The breadth is justified by the domain (scaffold → inspect → author → validate → record → compose → QA → export → publish assets), though a few (embed, feedback, frames) could arguably be folded into siblings.
Coverage of the demo lifecycle is nearly end-to-end: prereq check, demo init, trace import, selector inspection, schema/validation/lint, probe, all render stages, QA, and multiple export formats (gif, stills, frames, walkthrough, embed snippets). Minor gaps remain: no tool to read or edit an existing storyboard programmatically, no list_demos/delete/cleanup, and publishing is referenced only as an external workflow.