# synthesia-mcp

An MCP server that exposes the [Synthesia](https://www.synthesia.io) avatar video API to MCP clients such as Claude Desktop. Local stdio transport, API-key auth via environment, **draft-by-default rendering** so iteration never burns enterprise video quota.

This is a Phase 1 MVP focused on a clean capability layer; opinionated educational-video workflows live in a separate Claude skill that builds on top of these tools.

---

## Table of contents

- [What you can do with it](#what-you-can-do-with-it)
- [Requirements](#requirements)
- [Install & connect to Claude Desktop](#install--connect-to-claude-desktop)
- [First run](#first-run)
- [Configuration](#configuration)
- [Tool reference](#tool-reference)
- [Design notes](#design-notes)
- [Developer guide](#developer-guide)
- [Roadmap](#roadmap)

---

## What you can do with it

Once connected, you can ask Claude things like:

- *"List my Synthesia templates and show me the variables of the one called 'Lesson intro'."*
- *"Create a draft 2-scene video introducing photosynthesis with the Anna avatar."*
- *"Render that as a final video now."*
- *"How's video `abc-123…` doing? When it's done, make it public."*
- *"Upload `./diagram.png` to Synthesia and use it as the background of scene 2."*

The server holds the API key; Claude never sees it.

---

## Requirements

- **Node.js ≥ 20** (the server is shipped as an ESM bundle and uses the built-in `fetch`).
- **A Synthesia API key.** Enterprise plan recommended — the API is enterprise-tier on most plans. Find or create a key in Synthesia account settings: <https://app.synthesia.io/#/account>.
- An MCP-capable client. Claude Desktop is the primary target; the server works with any client that supports stdio MCP servers (e.g. Cursor, MCP Inspector).

---

## Install & connect to Claude Desktop

### 1. Get the server onto your machine

Clone or unzip this repository, then:

```bash
cd synthesia-mcp
npm install
npm run build
```

That produces `dist/index.js` — the executable the MCP client will run.

### 2. Register the server with Claude Desktop

Open your Claude Desktop config:

- **macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
- **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
- **Linux:** `~/.config/Claude/claude_desktop_config.json`

Add (or merge) an entry under `mcpServers`:

```json
{
  "mcpServers": {
    "synthesia": {
      "command": "node",
      "args": ["/ABSOLUTE/PATH/TO/synthesia-mcp/dist/index.js"],
      "env": {
        "SYNTHESIA_API_KEY": "your-synthesia-api-key-here"
      }
    }
  }
}
```

Use the **absolute path** to `dist/index.js` (Claude Desktop does not resolve `~` or relative paths).

### 3. Restart Claude Desktop

Quit fully and reopen. The Synthesia tools should appear in the tool picker. If they don't, check `~/Library/Logs/Claude/mcp*.log` (macOS) or the equivalent on your OS — the server logs to stderr and Claude Desktop captures it.

---

## First run

Once connected, ask Claude:

> *List my Synthesia templates.*

If you see your templates: you're set. If you see an `auth_error`: the API key is wrong or missing — double-check it in `claude_desktop_config.json` (no quotes inside the value, no trailing spaces) and restart Claude Desktop.

**Render your first video — for free:**

> *Create a draft 1-scene video titled "Hello" with the Anna avatar on a white_studio background. The avatar should say "Hello from Synthesia! This is my first API video."*

Claude will call `create_video` with `render_mode: "draft"` by default. The result includes a `videoId` and a note that this is a watermarked test render. Renders take a few minutes; ask Claude to check on it:

> *Is that video ready yet?*

Claude will call `get_video` and either tell you to wait, or hand you the download URL.

**Only when you're happy with the draft**, ask explicitly:

> *Render that as a final video.*

A `final` render counts against your account's video quota.

---

## Configuration

### Required: `SYNTHESIA_API_KEY`

Passed as an environment variable in `claude_desktop_config.json`'s `env` block. Per the MCP authorization specification, stdio servers retrieve credentials from the environment rather than implementing OAuth; that's exactly what this server does.

### Optional: `~/.synthesia-mcp/config.json`

Holds non-secret settings, primarily **your custom avatar and voice IDs** (Synthesia has no API endpoint for avatar/voice discovery, so any custom or personal avatar must be registered here to be discoverable via `list_avatars`).

Shape (see `config.example.json`):

```json
{
  "customAvatars": [
    { "id": "uuid-from-studio", "name": "My Personal Avatar", "notes": "..." }
  ],
  "customVoices": [
    { "id": "uuid-from-studio", "name": "My Custom Voice", "language": "English (UK)" }
  ],
  "defaults": {
    "aspectRatio": "16:9"
  }
}
```

To get a custom avatar ID: open <https://app.synthesia.io/#/actors>, hover the avatar, open the three-dot menu, and pick **Copy ID**.

Override the config path with `SYNTHESIA_MCP_CONFIG=/path/to/config.json` if you want it somewhere else.

### Refreshing the stock avatar/voice catalogs

The repo ships with a **small seed catalog** for offline use. To get the full lists from the live Synthesia documentation:

```bash
npm run refresh-catalog
```

That rewrites `catalog/avatars.json` and `catalog/voices.json`. Commit them. Re-run whenever Synthesia publishes new avatars or voices.

---

## Tool reference

All tools return a JSON object inside an MCP text content block. Errors carry `isError: true` with a structured `{ code, message, remediation, details? }` envelope.

### Discovery

| Tool | What it does |
| --- | --- |
| `list_avatars` | Lists stock avatars (bundled catalog snapshot) + custom avatars from config. Filter by `query`, `gender`, `version`, `source`. |
| `list_voices` | Lists stock voices (snapshot) + custom voices from config. Filter by `query`, `language`, `gender`. **Voices are optional** — when omitted, Synthesia uses its recommended voice for the chosen avatar. |
| `list_templates` | Lists STUDIO templates with their variable names. Paginated (`limit`/`offset`, `nextOffset` in response). Filter by `source: ["workspace", "synthesia"]`. |
| `get_template` | Returns a single template's full variable list — call this before `create_video_from_template` so you know the exact (case-sensitive) variable names. |

### Creation

| Tool | What it does |
| --- | --- |
| `create_video` | Creates a multi-scene video from raw scene objects (`avatar`, `background`, `scriptText` or `scriptAudioId`+`scriptLanguage`, `avatarSettings`, `backgroundSettings`). **Defaults to `render_mode: "draft"`** (free, watermarked test render). Pre-flight validates script tags (`<break time="Ns"/>`, `<sub alias="…">…</sub>` — anything else is rejected). |
| `create_video_from_template` | Renders a STUDIO template by passing `templateData` (variable values). The server fetches the template first and **validates every key against the declared variables** (variable names are case-sensitive — a mismatch returns the available-names list, not a render failure). Text values are HTML-entity-escaped by default (Synthesia requirement); pass `escapeText: false` only if you've pre-escaped. Supports `brandKitId` (incl. `"no_brand_kit"`). **Defaults to draft.** |

Both creation tools return a `videoId` immediately — Synthesia renders asynchronously, often in minutes. Check progress with `get_video`.

### Lifecycle

| Tool | What it does |
| --- | --- |
| `get_video` | Returns status (`in_progress` / `complete` / `error` / `rejected` / …) and, once complete, a **time-limited** download URL, duration, thumbnail. Call again to refresh the download URL if it expires. |
| `list_videos` | Lists videos in the account. Paginated, filterable by `source: ["workspace", "shared_with_me", "my_videos"]`. |
| `update_video` | Updates `title`, `description`, `visibility`, or `ctaSettings`. Main use: setting `visibility: "public"` to activate the share page on a completed video. |
| `delete_video` | **PERMANENT.** Double-gated: refuses unless `confirm: true` is passed. On the first call (without confirm) it returns the video's details so the user confirms against the real title, not just an ID. |

### Assets

| Tool | What it does |
| --- | --- |
| `upload_asset` | Uploads an image/video and returns an `assetId` usable as a scene background or as a template media variable value. Accepts `filePath` (local) **or** `url` (server downloads first). Allowed types: `image/jpeg`, `image/png`, `image/svg+xml`, `video/mp4`, `video/webm`. **GIF and WebP are rejected by Synthesia.** |
| `upload_script_audio` | Uploads narration audio (mp3 / `audio/mpeg`) for use as `scriptAudioId` (with `scriptLanguage`) in a scene. Note: Synthesia processes uploaded audio asynchronously — if a `create_video` call right after upload errors on the asset, wait briefly and retry. |

**Tip:** scene `background` fields and template media variables accept URLs directly. `upload_asset` is only needed when the media isn't publicly reachable or a stable asset ID is preferred.

---

## Design notes

A few decisions worth knowing about as a user:

- **Draft-by-default.** Every render path defaults to `render_mode: "draft"` (Synthesia `test: true`) — free, watermarked, no quota. `final` must be requested explicitly.
- **Server-side validation.** Mismatched template variable names, malformed script tags, and disallowed asset content types are caught locally before a request reaches Synthesia, so failures arrive instantly with the exact fix instead of after a wasted render.
- **Traceability.** Every created video gets a `callbackId` (auto-stamped `mcp/yyyy-mm-dd/title-slug` if you don't supply one). Useful for spotting MCP-originated videos in a shared workspace and for the planned Phase 4 webhook integration.
- **Token-frugal responses.** Tools return curated fields, not raw API payloads, to keep your context window healthy across long workflows.
- **Logging stays out of the way.** All logs go to stderr (stdout is reserved for the MCP protocol). The API key is registered with a redactor so it can never accidentally appear in a log line.
- **Rate-limit aware.** On HTTP 429, the server reads `RateLimit-Reset` and retries once after the indicated wait (capped). Synthesia Enterprise's tier limits are generous enough that you'll rarely see this.
- **No waiting/polling tool by design.** Renders can take minutes; blocking a chat turn that long is worse than a quick check-in. Use `get_video` when you're curious.
- **Hosted-readiness without speculation.** The credential layer is isolated behind a `CredentialProvider` interface; the tool registration is transport-agnostic. Switching to a hosted Streamable HTTP deployment with OAuth 2.1 later is a contained swap, not a rewrite.

---

## Developer guide

### Layout

```
src/
  index.ts                 # entry: wires credentials → client → tools → stdio transport
  credentials.ts           # CredentialProvider interface + EnvCredentialProvider
  config.ts                # local config loading (custom avatars/voices, defaults)
  catalog-loader.ts        # reads catalog/*.json
  logger.ts                # stderr-only logger with secret redaction
  errors.ts                # SynthesiaMcpError + error code enum
  synthesia/
    client.ts              # HTTP client: auth, two hosts, 429 retry, error mapping
    types.ts               # minimal API response types
  util/
    escape.ts              # templateData text entity escaping
    script-tags.ts         # break/sub tag validator
    tooling.ts             # ok/fail envelopes, guard wrapper, callbackId stamping
  tools/
    discovery.ts           # list_avatars, list_voices
    templates.ts           # list_templates, get_template
    videos.ts              # create_video, create_video_from_template, get_video, list_videos, update_video, delete_video
    assets.ts              # upload_asset, upload_script_audio
catalog/
  avatars.json             # seed snapshot; regenerate with npm run refresh-catalog
  voices.json              # seed snapshot
scripts/
  refresh-catalog.mjs      # regenerates the catalog files from docs.synthesia.io
test/
  smoke.mjs                # offline smoke test (no API key required)
```

### Build & test

```bash
npm install
npm run build       # tsc → dist/
npm run smoke       # offline: spawns the built server, calls list_avatars
```

The smoke test verifies all 12 tools are registered and that `list_avatars` returns catalog data — without making network calls, so no API key is required.

### Adding a new tool

1. Create or edit a module in `src/tools/`.
2. Define a zod schema for the inputs (it's both the validator and the source of the JSON Schema advertised to clients).
3. Wrap the handler with `guard(name, handler)` from `util/tooling.ts` to get consistent error envelopes.
4. Return data via `ok({...})` — keep the payload curated, not the raw API response.
5. Register the module from `src/index.ts`.
6. Add the tool name to `EXPECTED_TOOLS` in `test/smoke.mjs`.

### Authorization architecture

The MCP authorization specification draws a clean line between transports:

- **stdio (local):** retrieve credentials from the environment, no OAuth flow.
- **Streamable HTTP (remote):** OAuth 2.1 with PKCE, dynamic client registration, etc.

This server ships only the stdio path. The `CredentialProvider` interface in `credentials.ts` is the seam: a future hosted variant adds an HTTP transport in `index.ts` and a session-scoped provider implementing the same interface — no tool code changes. The Synthesia client always reads the key through the interface, never directly from `process.env`.

### Error model

All failures surface as `SynthesiaMcpError` instances. The `guard()` wrapper turns them into MCP tool results with `isError: true` and the structured `{ error: { code, message, remediation, details? } }` envelope. Error codes are deliberately few and category-level:

- `auth_error` — 401/403 from Synthesia
- `config_error` — missing env var or malformed config file
- `validation_error` — local pre-flight failed; **nothing was sent to Synthesia**
- `not_found` — 404 from Synthesia
- `rate_limited` — 429 after retry
- `moderation_rejected` — Synthesia content moderation
- `file_error` — local file unreadable, download failed, wrong type
- `upstream_error` — everything else from the Synthesia side

`remediation` is written for the LLM to self-correct in the next turn — concrete, specific, with the exact field name or available values where possible.

### Catalog refresh

`npm run refresh-catalog` fetches `https://docs.synthesia.io/reference/avatars.md` and `…/voices.md`, parses the markdown tables, and overwrites the JSON catalogs. The refresh script is intentionally simple: no schema migrations, no diffs — just replace and commit.

---

## Roadmap

This server is the **MCP layer** of a two-layer plan. Things deliberately *not* in v1:

- **Phase 4:** webhook tools (Synthesia fires `video.completed` / `video.failed` with captions, thumbnails, and your `callbackId` in the payload). Requires a public HTTP receiver; polling via `get_video` suffices for interactive authoring.
- **Phase 5:** translations, XLIFF export/import, dubbing — the localization surface is substantial and earns its own milestone.
- **Phase 6:** hosted Streamable HTTP deployment with OAuth 2.1, multi-account profiles, evals.

On top of this server, a separate **Claude skill** handles the educational workflow: eliciting audience, learning objectives, target duration (translated into a word-count budget), tone, and pedagogical structure; co-writing the script scene by scene; then delegating execution to these tools. The skill is opinionated and customizable per user; this server stays plug-and-play.

---

## License

MIT.