# ableton-agent

Drive [Ableton Live](https://www.ableton.com/live/) from an LLM agent. An MCP server exposes a running Live session — its tracks, devices, browser, clips, Arrangement and MIDI notes — through eight tools, plus the typed TypeScript library those tools are built on.

Everything runs locally over OSC, against your own open Live session.

## What an agent can do with it

The tool surface is deliberately small. Anything that is a *property* of something in Live — tempo, a track's volume, a device parameter, a clip's loop points — is read with `get_state` and written with `set_state`. Both address the same targets with the same arguments: nothing for the song, `track` (plus `isReturn` for a return track), `track` + `device`, or `track` + `slot` for a session clip. Six verb tools cover what isn't a property write.

| Tool | What it does |
| --- | --- |
| `get_state` | No arguments: the whole set (tempo, signature, key, transport, arrangement loop, every track with mixer/devices/clips, return tracks). `track`: that track plus color, arm and sends. `track` + `device`: the device's parameters with Live-formatted display values. `track` + `slot`: a clip with loop bounds and every MIDI note. `properties`: any extra LOM properties of the target. |
| `set_state` | Song: `tempo`, `timeSignature`, `key`, `playhead`, `arrangementLoop`. Track: `name`, `color`, `volume`, `pan`, `mute`, `solo`, `arm`, `sends`. Device: `parameters` (by name or index), `randomizeMacros`. Clip: `name`, `color`, `looping`, `loopStart`, `loopEnd`, `notes` (replace all). Any target: `properties` for arbitrary LOM properties. Several fields per call; fields that don't belong to the target are rejected before anything is written. |
| `create` | `midi_track`, `audio_track`, `return_track`, `scene`, `midi_clip`, `audio_clip`, `clip_duplicate`, `arrangement_clip` (copy a session clip onto the timeline). |
| `delete` | `track`, `scene`, `clip`, `device`, `arrangement_clip`. |
| `transport` | `play`, `stop`, `play_arrangement` (Back to Arrangement + play), `fire_scene`, `fire_clip`, `stop_clips`. |
| `browse` | Reflect a browser category (instruments, audio_effects, plugins, drums, …) or any path from a previous result, one level at a time. |
| `load` | Load a browser item onto a track's chain, onto an existing device (replacing it), or onto one Drum Rack pad. |
| `edit_notes` | Incremental MIDI edits on a clip: `add`, `remove` (by range), `transpose`, `quantize`. |

`get_state` with no arguments is the one an agent should reach for first. A typical exchange:

```jsonc
get_state   {}                                                  // see the set
create      { "type": "midi_track", "name": "Bass" }            // → "Created MIDI track at index 3 named "Bass"."
browse      { "category": "instruments" }                       // find something to load
load        { "track": 3, "path": "live_app browser instruments children 0" }
create      { "type": "midi_clip", "track": 3, "slot": 0, "lengthBeats": 4 }
set_state   { "track": 3, "slot": 0, "notes": [{ "pitch": 36, "start": 0, "duration": 1 }] }
set_state   { "track": 3, "volume": 0.7, "sends": { "0": 0.3 } }
set_state   { "track": 3, "device": 0, "parameters": { "Filter Freq": 0.4 } }
transport   { "action": "fire_clip", "track": 3, "slot": 0 }
```

## Requirements

- **Ableton Live 11** — developed and verified against Live 11. Any edition; Max for Live is not required.
- **[Bun](https://bun.com)** 1.3+
- macOS — what this is developed and tested on. Nothing here is macOS-specific in principle, but Windows is untested; the equivalent Remote Scripts folder there is `Documents\Ableton\User Library\Remote Scripts`.

## Setup

Install dependencies:

```bash
bun install
```

Install the **AbletonAgent remote script** — this is the transport, so nothing works without it. Symlink or copy [`packages/ableton-agent/remote-script/AbletonAgent`](packages/ableton-agent/remote-script/AbletonAgent) into Live's User Library:

```bash
bun run --filter ableton-agent install-remote-script
```

Pass `--target-dir <path>` if your Live User Library lives somewhere else.

Then enable **AbletonAgent** as a Control Surface in Live's **Preferences → Link/Tempo/MIDI**. Once enabled it's active for every project you open. Live only loads remote scripts at startup, so restart Live after installing or editing it.

Sanity-check that Live is reachable:

```bash
bun run --filter ableton-agent check-agent
```

## Wiring it into an MCP client

The server speaks MCP over stdio, so it's launched by the client rather than left running in a terminal.

Claude Desktop — add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "ableton-agent": {
      "command": "bun",
      "args": ["run", "/absolute/path/to/ableton-agent/packages/ableton-agent-mcp/index.ts"]
    }
  }
}
```

Claude Code:

```bash
claude mcp add ableton-agent -- bun run /absolute/path/to/ableton-agent/packages/ableton-agent-mcp/index.ts
```

To run it directly while debugging: `bun run --filter ableton-agent-mcp start`.

## Configuration

Ports are fixed on both sides by default; override with env vars if you've changed either end.

| Env var | Default | Transport |
| --- | --- | --- |
| `ABLETON_AGENT_ADDRESS` | `127.0.0.1` | AbletonAgent |
| `ABLETON_AGENT_LOCAL_PORT` | `9011` | AbletonAgent |
| `ABLETON_AGENT_REMOTE_PORT` | `9010` | AbletonAgent |
| `ABLETON_AGENT_TIMEOUT_MS` | `8000` | AbletonAgent |
| `ABLETON_MAX_ADDRESS` | `127.0.0.1` | Max bridge |
| `ABLETON_MAX_LOCAL_PORT` | `9001` | Max bridge |
| `ABLETON_MAX_REMOTE_PORT` | `9000` | Max bridge |
| `ABLETON_MAX_TIMEOUT_MS` | `8000` | Max bridge |

If you change the AbletonAgent ports, change them in [`remote-script/AbletonAgent/__init__.py`](packages/ableton-agent/remote-script/AbletonAgent/__init__.py) too — `OSC_LISTEN_PORT` must match `ABLETON_AGENT_REMOTE_PORT`, and `OSC_REPLY_PORT` must match `ABLETON_AGENT_LOCAL_PORT`.

### Optional: the Max for Live bridge

[`packages/ableton-agent/max/ableton-agent.amxd`](packages/ableton-agent/max/ableton-agent.amxd) is an auxiliary second transport (ports 9000/9001) offering generic get/set/call access to the Live Object Model. It's useful for raw poking and diagnostics, and it can run side by side with the remote script — but nothing in the library or MCP server needs it. Load it onto any track in your set if you want it.

## Using the library directly

`packages/ableton-agent` is usable on its own, without MCP — a typed façade (`Live`/`Song`/`Track`/`Clip`/`Device`) over the OSC transports. Every method returns a promise that resolves once Live has actually executed the change, so a resolved call means it happened, and a bad path rejects with Live's own error.

```ts
import { Live } from "ableton-agent";

const live = new Live({
  max: { address: "127.0.0.1", localPort: 9001, remotePort: 9000 },
  remote: { address: "127.0.0.1", localPort: 9011, remotePort: 9010 },
});
await live.connect();

console.log(await live.getProjectState());

const track = live.track(0);
await track.setVolume(0.85);
await track.clip(0).addNotes([{ pitch: 60, start: 0, duration: 1, velocity: 100 }]);

live.song.onIsPlayingChange((playing) => console.log(playing ? "playing" : "stopped"));
await live.song.play();
```

## Repo layout

A Bun workspace with two packages:

- [`packages/ableton-agent`](packages/ableton-agent) — the TypeScript library plus the two OSC transports: the Python `AbletonAgent` remote script (primary — everything goes through it) and the Max for Live `bridge.js` device (auxiliary, generic get/set/call only).
- [`packages/ableton-agent-mcp`](packages/ableton-agent-mcp) — the MCP server: the eight tools above, built on the library. [`.claude/references/`](.claude/references) holds the Live Object Model and Remote Script API references the tool descriptions point agents at.

Run a package script from the root with `--filter`:

```bash
bun run --filter ableton-agent check-agent
bun run --filter ableton-agent-mcp start
```

## Testing

There are two kinds of tests, and the difference matters:

```bash
# Hermetic — no Ableton needed, safe to run anywhere.
bun test packages/ableton-agent/live.test.ts \
         packages/ableton-agent/liveBridge.test.ts \
         packages/ableton-agent-mcp/index.test.ts
```

These exercise the library and every MCP tool against an in-memory stand-in for Ableton ([`testSupport/fakeAbleton.ts`](packages/ableton-agent-mcp/testSupport/fakeAbleton.ts)) that speaks the real OSC wire protocol over real sockets.

```bash
# Everything, including the real-session integration suite.
bun test
```

Plain `bun test` also picks up [`index.real.test.ts`](packages/ableton-agent-mcp/index.real.test.ts), which drives **a real, running Ableton Live session**. There's no opt-in flag and no skipping: every test always attempts its real call and reports a genuine pass or fail, so **`bun test` requires Ableton to be open and will start real, audible playback**. If Live isn't reachable, those tests fail with a message explaining what's missing — use `check-agent` to debug connectivity.

All its mutations are confined to a scratch MIDI track it creates and deletes; the global properties it touches (tempo, time signature, key, playhead) are saved and restored.

## License

MIT — see [LICENSE](LICENSE).