Skip to main content
Glama

Claude_AV_MCP

Sandbox for audio-visual MCP experiments aimed at technical artists: Claude builds instruments and visuals and conducts live AV sets across Ableton Live, Max/MSP and a visual engine (TBD) — first on a MacBook Pro M5, later on a cheap Linux laptop with an all-open-source stack.

How it fits together

Server

Role

Source

ableton

Edit the Live set: tracks, clips, notes, devices, transport

ahujasid/ableton-mcp (upstream, unchanged)

maxmsp

Edit Max patches: add/connect objects, read docs

tiianhk/MaxMSP-MCP-Server (upstream, unchanged)

av-conductor

Shared scene (energy, section, palette, cues, ramps) → OSC to every program

this repo, src/av_mcp/

Upstream servers author; the conductor directs; beat-accurate sync and audio reactivity stay in the realtime plane (Ableton Link, OSC between programs) where Claude never blocks them. Details: docs/ARCHITECTURE.md.

Related MCP server: ableton-mcp

Quick start (Mac)

# 1. Python env for the conductor (needs uv: https://docs.astral.sh/uv/)
uv sync --extra dev
uv run pytest

# 2. Upstream servers → vendor/ (or symlink your existing clones there)
scripts/fetch_upstream.sh

# 3. Watch the conductor's output with no apps running
cp config/av.example.toml config/av.toml      # set [targets.monitor] enabled = true
uv run av-osc-monitor --port 7499

Then open Claude Code in this folder — .mcp.json registers all three servers. For Claude Desktop, merge config/claude_desktop_config.example.json into its config (absolute paths).

In Max, open patches/max/av_receiver.maxpat; in Pd, patches/pd/av_receiver.pd. Try:

"Set the scene to section intro with energy 0.2, then ramp energy to 1 over 20 seconds."

Layout

src/av_mcp/          av-conductor MCP server (scene, OSC targets, ramps, cues)
config/              av.example.toml (targets/ports/cues), Claude Desktop example
patches/max|pd/      OSC receiver patches → [r av-energy], [r av-section] …
visuals/             visual engine candidates + receiver sketches (TBD)
prompts/             prompt library
docs/                PLAN.md · ARCHITECTURE.md · LINUX_PORT.md
scripts/             fetch_upstream.sh
vendor/              upstream MCP servers (gitignored)

Roadmap

docs/PLAN.md: Phase 0 plumbing → 1 audio/control loop → 2 pick the visual engine → 3 performance tooling → 4 Linux port (Pd, Ardour/SuperCollider, PipeWire, open-source visuals).

Privacy note

ableton-mcp sends anonymous telemetry by default. The configs here set ABLETON_MCP_DISABLE_TELEMETRY=true and ABLETON_MCP_DISABLE_DATASET=true.

Available Tools

7 tools
get_stateC

Current scene, saved cue names, targets, and running ramps.

Inputs: no named parameters. Returns: result

rampB

Glide a numeric value over time (a build, a fade). Returns immediately.

param: "energy", "bpm", or "macro:" (e.g. "macro:brightness").

Inputs: to (required), param (required), seconds (required). Returns: result

recall_cueB

Jump to a saved cue (from config/av.toml or save_cue) and send /av/cue .

Inputs: name (required). Returns: result

resend_allA

Re-broadcast the whole scene, e.g. after restarting Max or the visual engine.

Inputs: no named parameters. Returns: result

save_cueB

Snapshot the current scene under a name (kept until the server restarts).

Inputs: name (required). Returns: result

send_oscA

Escape hatch: send a raw OSC message, to all targets or the named ones.

Use for program-specific messages the scene model doesn't cover (e.g. "/visuals/layer/2/opacity" 0.5).

Inputs: address (required), args, targets. Returns: result

set_sceneA

Change the shared scene and broadcast it to every target over OSC.

energy and macros are 0..1. palette is a list of hex colors. Only fields you pass are changed. Note: bpm here is informational for the visuals; to change Live's tempo also call ableton-mcp's set_tempo (or use Ableton Link so everything follows Live).

Inputs: bpm, key, energy, macros, +2 more. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 7 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: save/recall cues, raw OSC escape, full rebroadcast, state query, scene update, and time-based ramping. No two tools overlap in function or would cause misselection.

    Naming Consistency4/5

    Names are consistently snake_case and mostly follow a verb_noun pattern (save_cue, recall_cue, send_osc, get_state, set_scene, resend_all). The lone 'ramp' breaks the pattern slightly, but the overall convention is clear.

    Tool Count5/5

    Seven tools are well-scoped for an AV conductor, covering scene management, cue handling, OSC messaging, and ramping without bloat. Each tool earns its place.

    Completeness3/5

    Core lifecycle is present (create/read/update for scenes and cues), but notable gaps exist: no tool to delete or overwrite cues explicitly, and no way to cancel or stop an active ramp. These are common AV operations that agents cannot perform directly.

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables natural language control over Ableton Live for generating musical patterns, melodies, and full song arrangements. It also provides tools for sample searching and mixing assistance through an OSC-based connection with Claude Desktop.
      1
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables Claude Code to control Ableton Live directly through a Max for Live device, letting users read set state, search samples, place clips, build Drum Racks, adjust device parameters, and generate .als project files from sample packs without launching Live.
      MIT