# hydra-mcp

An [MCP](https://modelcontextprotocol.io) server that lets an AI agent live-code
[Hydra](https://hydra.ojack.xyz), the browser video synth, alongside a person, and send the
result to other software as a real video source.

- **Two targets.** The agent can drive Hydra in its own local performer page, or in an ordinary
  Hydra tab such as hydra.ojack.xyz.
- **Dry-run first.** Every piece of code runs in a hidden second Hydra instance before it touches
  the live canvas. A misspelled function or broken shader comes back as a real error, with the
  live output untouched.
- **Shared piece, safe ownership.** The agent and a person write into one shared composition,
  and neither can silently overwrite the other.
- **Sees its own output.** The agent takes screenshots, so it can check what it made.
- **Performance tools.** Crossfades, custom GLSL functions, image and video sources, camera and
  microphone input, a sketch library, and share links.
- **Native video out.** Syphon (macOS) and NDI, for TouchDesigner, OBS, Resolume and other hosts.

```
MCP client ──stdio──> server/hydra_mcp.py ──HTTP──> deck/hydra_deck.py ──WebSocket──> Hydra tab(s)
(your agent)          (thin MCP proxy)              (long-lived, 127.0.0.1:8140)       │
                                                                                        └─ frames ─> out/hydra_out.py ─> Syphon / NDI
```

---

## Contents

1. [Requirements](#requirements)
2. [Install](#install)
3. [Run it](#run-it)
4. [Connect your MCP client](#connect-your-mcp-client)
5. [The performer page](#the-performer-page)
6. [Tools](#tools)
7. [Working with the tools (for agents)](#working-with-the-tools-for-agents)
8. [Driving an existing Hydra tab](#driving-an-existing-hydra-tab)
9. [Video out: Syphon and NDI](#video-out-syphon-and-ndi)
10. [Configuration](#configuration)
11. [How it works](#how-it-works)
12. [Security model](#security-model)
13. [Troubleshooting](#troubleshooting)
14. [Known limitations](#known-limitations)
15. [Project layout and development](#project-layout-and-development)
16. [Licence](#licence)

---

## Requirements

| Needed for | What |
|---|---|
| Everything | [uv](https://docs.astral.sh/uv/). It installs the right Python for you: 3.10+ for the deck, 3.12 for video out |
| The Hydra page | A Chromium-based browser (Chrome, Edge, Brave, Arc). Firefox and Safari can run the page, but not video out |
| Syphon out | macOS |
| NDI out | macOS, Windows or Linux. Receivers need NDI support (TouchDesigner, OBS with the NDI plugin, NDI Tools, …) |
| Driving hydra.ojack.xyz | Chrome, with the unpacked extension in `link/` |

No Node.js is needed to run anything. The browser dependencies are vendored.

## Install

```bash
git clone https://github.com/brandon-stargrave/hydra-mcp.git
cd hydra-mcp
uv sync                                   # deck + MCP server
```

Video out is optional and has its own environment. Install only the outputs you want:

```bash
cd out
uv sync --extra syphon --extra ndi        # macOS: both
uv sync --extra ndi                       # Windows / Linux: NDI only
cd ..
```

## Run it

**1. Start the deck** and leave it running. It is the process that holds the piece.

```bash
uv run deck/hydra_deck.py
```

**2. Open the performer page** in a browser window you can see:
[http://127.0.0.1:8140/](http://127.0.0.1:8140/). Keep it visible, because browsers throttle
background tabs (see [Troubleshooting](#troubleshooting)).

**3. Connect your MCP client** ([next section](#connect-your-mcp-client)). The client starts
`server/hydra_mcp.py` itself.

**4. (Optional) Start video out:**

```bash
cd out && uv run --extra syphon --extra ndi hydra_out.py --ndi
```

Then have the agent call `hydra_video_out(action="start")`.

## Connect your MCP client

The MCP server is a stdio process. Every MCP client wants the same three things: a command, its
arguments, and optional environment variables. Use **absolute paths**, because clients don't run
your shell and inherit no `PATH`. Find uv with `which uv`.

Most MCP clients take an `mcpServers` entry in this shape. Your client's docs say where its
config file lives:

```json
{
  "mcpServers": {
    "hydra": {
      "command": "/absolute/path/to/uv",
      "args": ["--directory", "/absolute/path/to/hydra-mcp", "run", "server/hydra_mcp.py"]
    }
  }
}
```

If your client has an `mcp add` command instead, give it the same command and arguments.

Restart the client afterwards. It should list 19 `hydra_*` tools. The server finds the deck at
`http://127.0.0.1:8140` unless told otherwise (see [Configuration](#configuration)). If the deck
isn't running, every tool returns `{"ok": false, "error": "unreachable"}` with the command to
start it.

## The performer page

| URL | What |
|---|---|
| `http://127.0.0.1:8140/` | Performer view: the canvas, a code editor over it, and a status line |
| `http://127.0.0.1:8140/out` | Clean full-screen output for a second screen, projector or browser source. Follows the performer page by default |
| `?role=primary` / `?role=mirror` | Choose which tab answers the agent (see [Tabs and roles](#tabs-and-roles)) |
| `?audio=1` | Turn on microphone input so `a.fft` works. The browser asks for permission |
| `?w=1920&h=1080` | Render size (default 1280×720) |
| `/out?hud=1` | Show the status line on `/out` |

The editor shows the shared composition as `// @block <name>` sections.

| Keys | Action |
|---|---|
| Ctrl+Enter (or Cmd+Enter) | Run the block under the cursor. It is dry-run first and saved to the document as yours |
| Shift+Ctrl+Enter | Run every block |
| Shift+Ctrl+H | Hide or show the editor and status line |
| Shift+Ctrl+R | Pull in changes the agent made while you had unsent edits. The editor never types over you |

## Tools

All tools return JSON (and `hydra_look` also returns an image). Failures come back as
`{"ok": false, "error": "...", "say": "...", "hint": "..."}` rather than exceptions.

### Reading

| Tool | Arguments | Returns |
|---|---|---|
| `hydra_state` | — | Connected tabs (role, adapter, visible, fps, size, speed/bpm, mic permission, live sources), blocks with authors and the outputs they draw into, recent runtime warnings |
| `hydra_document` | — | Every block (name, code, author) and the whole piece as text. For an existing Hydra tab, also `tab_editor`: that tab's own editor text |
| `hydra_look` | `max_width=640` | A PNG screenshot of what is on screen. Refuses honestly if the tab is hidden |
| `hydra_reference` | `name=""`, `type=""` | Hydra's transform functions read live from the engine: all names by type plus the primer, one function's inputs and defaults, or every function of a type |

### Writing the piece

| Tool | Arguments | Does |
|---|---|---|
| `hydra_block` | `name`, `code`, `validate=true`, `take=false` | Writes, dry-runs and runs one named block. Other blocks keep playing |
| `hydra_crossfade` | `name`, `code`, `seconds=4`, `validate=true`, `take=false` | Like `hydra_block`, but fades to the new code instead of cutting |
| `hydra_drop` | `name`, `take=false` | Removes a block. Blanks outputs only it drew into, and re-runs any other block on the same output |
| `hydra_sketch` | `code`, `validate=true`, `take=false` | Replaces the whole piece. Split it with `// @block <name>` lines |
| `hydra_define` | `name`, `type`, `glsl`, `inputs=[]`, `take=false` | Adds a custom GLSL transform (Hydra's `setFunction`), compiled in a throwaway chain first |

### The instrument

| Tool | Arguments | Does |
|---|---|---|
| `hydra_render` | `output="all"` | Shows `o0`–`o3` full-screen, or all four as a grid |
| `hydra_transport` | `speed`, `bpm`, `width`+`height` (any subset) | Global clock and canvas size |
| `hydra_source` | `slot` (`s0`–`s3`), `kind` (`image` / `video` / `cam` / `screen` / `clear`), `target` | Loads an external source. Reports `live`, `pending_permission` or why it failed |
| `hydra_audio` | `enable`, `bins`, `smooth`, `cutoff`, `scale`, `show` | Reads or tunes `a.fft`. `enable=true` reloads the page with the microphone on |
| `hydra_video_out` | `action` (`start` / `stop` / `status`), `fps=30` | Streams the canvas to Syphon/NDI and reports the fps measured at both ends |
| `hydra_hush` | — | Panic: blank everything and reset speed. The document is archived first |

### Library and sharing

| Tool | Arguments | Does |
|---|---|---|
| `hydra_save` | `name`, `tags=[]`, `overwrite=false` | Saves the piece, with authors and a thumbnail, to `sketches/` |
| `hydra_load` | `name`, `take=false`, `validate=true` | Loads a saved piece, keeping each block's original author |
| `hydra_sketches` | `query=""` | Lists saved pieces, filtered by name or tag |
| `hydra_share_url` | — | A hydra.ojack.xyz link that opens the piece in the official editor. Built locally; nothing is uploaded |

Each tool's own description (what your client shows the agent) goes into more detail. A primer
on Hydra idioms is at [`knowledge/hydra-primer.md`](knowledge/hydra-primer.md) and is returned
by `hydra_reference()`.

## Working with the tools (for agents)

**The loop that works:**

1. **`hydra_state`**: is a tab connected and `visible`? Is anything erroring?
2. **`hydra_document`**: what is already playing, and who wrote it? Extend the piece; don't talk
   over it.
3. **`hydra_block`**: write one named block. If it's refused, read `errors`, fix, try again.
   Don't pass `validate=false` just to get past an error.
4. **`hydra_look`**: look before you describe anything. Code that runs cleanly can still render
   black, blown out or frozen.

**Conventions:**

- **Outputs and blocks.** There are four outputs, `o0`–`o3`. A bare `.out()` means `.out(o0)`.
  Give each output its own block (`base` → o0, `texture` → o1, `mix` → o2), and use a `show`
  block holding `render(o2)`.
- **Values.** Plain numbers are compiled into the shader. For motion, use functions
  (`() => Math.sin(time)`) or arrays (`[1, 2, 4].fast(0.5)`).
- **Feedback.** Read an output back into itself: `src(o0).scale(1.01).blend(o1, 0.1).out(o0)`.
- **Big changes mid-set.** Use `hydra_crossfade`. `hydra_hush` is for emergencies only.
- **Permissions.** Never assume someone clicked Allow. Camera and microphone report
  `pending_permission` until they do.
- **`take=true`.** It exists only for when the person playing asked you to take over their work.

**Example session:**

```text
hydra_state()                                     → one primary tab, visible, 60 fps, no blocks
hydra_sketch("// @block base\nosc(14, 0.04, 1.3).kaleid(6).out(o0)\n// @block show\nrender(o0)")
hydra_look()                                      → check it
hydra_block("texture", "noise(2.5, 0.1).modulateScale(osc(4), 0.5).out(o1)")
hydra_block("mix", "src(o0).modulate(o1, 0.12).out(o2)")
hydra_block("show", "render(o2)")
hydra_crossfade("base", "voronoi(8, 0.3, 0.2).color(1, 0.4, 0.8).out(o0)", seconds=6)
hydra_save("first-set", tags=["kaleid", "modulate"])
```

## Driving an existing Hydra tab

The agent can also play inside a normal Hydra editor tab, such as
[hydra.ojack.xyz](https://hydra.ojack.xyz). The agent's blocks are written into that tab's own
editor as `// @block` sections, and the tab's own code stays in charge.

Chrome won't let a public HTTPS page reach `127.0.0.1` without a prompt, and Safari blocks it
outright. So the reliable route is the small unpacked Chrome extension in `link/`: its
background worker holds the connection to the deck.

**Install the extension (once):**

1. In Chrome, open `chrome://extensions` and turn on **Developer mode** (top right).
2. Click **Load unpacked** (top left) and select the repository's **`link` folder**. Select the
   folder itself, not a file inside it.
3. Copy the **ID** shown on the new "Hydra Deck Link" card.
4. Add it to `~/.hydra-deck/config.json`, which allows the extension to connect:

   ```json
   { "extension_ids": ["<the id you copied>"] }
   ```

5. Restart the deck.

**Use it:**

1. Open (or reload) `https://hydra.ojack.xyz`. The extension attaches automatically; on any other
   Hydra page, click the extension's toolbar icon. Close the site's intro panel so you can see the
   editor.
2. Make that tab the one the agent drives. Either close the deck's own page, or open the deck
   page as `http://127.0.0.1:8140/?role=mirror`.
3. `hydra_state` now shows a primary tab with `adapter: "ojack"`.

**What changes for this kind of tab:**
- `hydra_document` includes `tab_editor`, that tab's live editor text.
- `hydra_sketch` refuses (`tab_has_other_code`) if the editor holds code the agent didn't write.
- `hydra_drop` restores outputs from the tab's own editor sections.

**After changing anything in `deck/page/`,** run `tools/sync-link.sh`. Then press ↻ on the
extension card *and* reload the Hydra tab. Tabs that were already open keep the old scripts.

A zero-install alternative (experimental) is to paste this into the Hydra tab's DevTools console.
Chrome asks once for local network access; Safari can't do this.

```js
(s => { s.src = 'http://127.0.0.1:8140/bridge.js'; document.head.appendChild(s); })(document.createElement('script'))
```

## Video out: Syphon and NDI

`hydra_video_out(action="start")` streams the primary tab's canvas, frame by frame, to
`out/hydra_out.py`. That process publishes it as:

| Output | Platform | Receivers see it as | Notes |
|---|---|---|---|
| Syphon | macOS | `Python:Hydra` (TouchDesigner shows senders as App:Server) | Published flipped for GL receivers. Use `--no-flip` if yours shows it upside down |
| NDI | macOS / Windows / Linux | `<MACHINE NAME> (Hydra)` | Pass `--ndi`. Reaches other machines on the network |

```bash
cd out
uv run --extra syphon --extra ndi hydra_out.py --ndi          # both
uv run --extra ndi hydra_out.py --ndi --no-syphon              # NDI only
uv run hydra_out.py --help                                     # --name, --no-flip, …
```

`hydra_video_out` reports measurements, not the fps you asked for:
- `page_after_2s.sent_fps` and `dropped`: what the tab actually sent.
- `out_after_2s.fps`: what was published.
- `syphon_clients` and `ndi_connections`: whether anything is receiving.

Frames are **dropped, never queued**, when the receiver falls behind, so latency stays low.

**In TouchDesigner:**
- **Syphon.** Add a *Syphon Spout In TOP* and pick `Python:Hydra`. Plain `Hydra` matches
  nothing and leaves the checkerboard.
- **NDI.** Add an *NDI In TOP* and choose the source in its **Name** menu.
- **Throughput.** Measured on an Apple Silicon Mac at 1280×720, both paths delivered 30 fps to
  TouchDesigner.

Video out requires the primary tab to be in a Chromium browser and **visible**.

## Configuration

`~/.hydra-deck/config.json` is read by the deck, the MCP server and video out. Environment
variables override it. Every key is optional:

| Key | Env | Default | What |
|---|---|---|---|
| `port` | `HYDRA_DECK_PORT` | `8140` | Deck port |
| `token` | `HYDRA_DECK_TOKEN` | — | If set, the control API requires `X-Hydra-Token` (the MCP server sends it automatically) |
| — | `HYDRA_DECK_URL` | `http://127.0.0.1:<port>` | Where the MCP server looks for the deck |
| `allowed_origins` | — | `["https://hydra.ojack.xyz"]` is always allowed | Extra browser origins allowed to connect, e.g. a self-hosted Hydra editor |
| `extension_ids` | — | — | IDs of the `link/` extension allowed to connect |
| `out_port` | `HYDRA_OUT_PORT` | `8141` | Video out port |
| — | `HYDRA_DECK_HOME` | `~/.hydra-deck` | Where config, the live document and hush archives live |

**State on disk:**
- `~/.hydra-deck/document.json` holds the current piece. A restarted deck replays it into a newly
  opened page.
- `~/.hydra-deck/hushed-*.json` holds what `hydra_hush` cleared.
- `sketches/` holds the saved library. It is git-ignored.

## How it works

- **Why two processes.** An MCP stdio server lives only as long as one client session, and a
  performance must not end when a chat restarts. So the long-lived **deck** holds the document
  and the browser connections. The MCP server only forwards requests.
- **Tabs and roles.** <a name="tabs-and-roles"></a> Every connected tab is a **primary** or a
  **mirror**:
  - The primary validates, runs code, takes screenshots and streams video.
  - Mirrors run the same code and never answer.
  - Who is primary: an explicit `?role=primary` wins. Otherwise it's the longest-connected tab
    that didn't ask to be a mirror.
- **Adapters.** `bridge-core.js` is the same in every tab; a small adapter says where Hydra lives:
  - `deck` is this project's page.
  - `ojack` is the official editor, CodeMirror 5 or 6.
  - `generic` is any page with a running Hydra, such as Strudel's `initHydra`.
- **Validation.**
  - **Why shaders need a separate check.** Hydra's shader failures are silent: its bundled
    renderer never checks compile status, so a broken shader just renders black.
  - **The dry run.** Code runs inside `with (synth) { … }` against a second, non-global Hydra on
    a small offscreen canvas.
  - **The shader check.** Every fragment shader generated along the way is compiled again with
    raw WebGL, which returns the real error log.
  - **Custom functions.** Functions defined with `setFunction` are replayed into the validator,
    so blocks that use them validate too.
- **Ownership.** Blocks record `author: "agent"` (anything through MCP) or `author: "human"`
  (typed in the deck page's editor). Refusal codes are `human_block` and `human_blocks`.
  Nothing arriving over HTTP can write as `human`.

## Security model

Running code in the tab is arbitrary JavaScript in someone's browser, so the deck is strict:

- **Loopback only.** Everything binds to `127.0.0.1`.
- **Control API.** `/deck/*` refuses any request that carries an `Origin` header, which means
  every web page. Every route requires `Host: 127.0.0.1:<port>` (or localhost), which defeats
  DNS rebinding. An optional token adds a shared secret.
- **WebSocket.** It accepts only allow-listed browser origins: the deck itself,
  `https://hydra.ojack.xyz`, your `allowed_origins`, and your `extension_ids`.
- **Tabs only report.** A connected tab can never make another tab run code. The one exception
  is the deck page's own editor, on the deck's own origin.
- **Extension scope.** The extension's worker only connects to `ws://127.0.0.1:8140`. A page
  can't choose where it connects.

## Troubleshooting

| Symptom | Cause and fix |
|---|---|
| Every tool returns `unreachable` | The deck isn't running: `uv run deck/hydra_deck.py` |
| `no_page` | No Hydra tab is connected. Open `http://127.0.0.1:8140/` or attach a Hydra tab |
| `hydra_look` says the tab is hidden, or `no_frame` | Background tabs and minimized windows are throttled. Bring the primary tab's window to the front |
| Screenshots come out black where you expected content | Empty outputs are transparent, and screenshots flatten them onto black, as on screen. Check that the output you `render` is being drawn |
| A block is refused with a GLSL log | That is the dry run working. Fix the shader. Note that `sum()` doesn't compile in hydra-synth 1.4.0; use `add` |
| `human_block` / `human_blocks` | The block was written by the person at the page. Use another name, or `take=true` if they asked |
| `no_spare_outputs` on crossfade | A fade borrows two outputs no other block uses. Free one with `hydra_drop`, or cut with `hydra_block` |
| Camera shows `pending_permission` | The browser is waiting for someone to click Allow |
| Microphone `denied` | Allow the microphone for `127.0.0.1:8140` in the browser's site settings |
| Image or video source `not_live` | The URL must allow cross-origin use (CORS) |
| The extension tab never appears in `hydra_state` | Its ID isn't in `extension_ids`, or the tab was open before the extension loaded (reload it). Look for `refused a websocket from origin` in the deck's output |
| Changes to page code don't take effect in the extension tab | Run `tools/sync-link.sh`, press ↻ on the extension card, **and** reload the tab |
| TouchDesigner Syphon In shows a checkerboard | Choose sender `Python:Hydra`, not `Hydra` |
| Syphon image upside down in your receiver | Restart video out with `--no-flip` |
| `out_not_running` | Start `out/hydra_out.py` first |
| High `dropped` count in video out | Lower the resolution (`hydra_transport(width=…, height=…)`) or the fps |

## Known limitations

- **Browsers.** Video out needs a Chromium browser (it uses `MediaStreamTrackProcessor`).
- **Mirrors.** Mirrors have their own clock and microphone, so audio-reactive visuals can differ
  between tabs. They also don't open their own camera or screen capture.
- **Screen capture.** Browsers require a click inside the page, so an agent usually can't start
  it remotely.
- **Deleting in the editor.** Deleting a block's text in the deck page's editor doesn't remove
  it from the document. Use `hydra_drop`.
- **Extensions.** Code that calls `loadScript` (Hydra extensions) skips the dry run, and says so.
- **Paste route.** The `/bridge.js` console route is experimental; the extension is the
  supported path.

## Project layout and development

```
server/hydra_mcp.py        MCP server (stdio)
deck/hydra_deck.py         the deck: page server, WebSocket hub, control REST, document
deck/page/                 browser code (AGPL): bridge-core.js, frames.js, adapters/, index.html, out.html
deck/page/vendor/          hydra-synth 1.4.0 (unmodified) and the CodeMirror 6 bundle
link/                      Chrome extension (copies of the bridge files, kept in sync by tools/sync-link.sh)
out/hydra_out.py           video out (own pyproject, Python 3.12)
knowledge/hydra-primer.md  Hydra idioms, returned by hydra_reference()
tools/                     build-cm6.sh (rebuild the editor bundle), sync-link.sh
sketches/                  your saved library (git-ignored)
```

- **Rebuild the editor bundle:** `tools/build-cm6.sh`. This needs Node.js and npm, and pins
  CodeMirror and esbuild versions.
- **After editing `deck/page/`:** run `tools/sync-link.sh`, then reload the extension and its tabs.
- **The block parser exists twice:** `parse_blocks()` in `deck/hydra_deck.py` and `blocksOf()` in
  `deck/page/adapters/deck.js`. Change both or neither.
- **Quick checks:**
  - `node --check deck/page/*.js deck/page/adapters/*.js`
  - `uv run python -m py_compile deck/hydra_deck.py server/hydra_mcp.py`

## Licence

The Python code is **MIT** (`LICENSE`). The browser code in `deck/page/` and `link/` is
**AGPL-3.0-or-later**, because it runs alongside hydra-synth, which is AGPL-3.0. See
[`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md) for everything bundled or installed,
including the NDI runtime's terms.

Hydra is by Olivia Jack and contributors. NDI® is a registered trademark of Vizrt NDI AB.