composing-sidekick

composing-sidekick lets an AI assistant compose with you inside Ableton Live, using your own past music as its main material. It reads the MIDI in your Live Sets, learns your phrases and habits, and answers what you are playing with ideas built from them. It writes those ideas only to tracks it owns, so you listen and keep or drop each one.

It is for producers who work in Ableton Live and want a partner that plays back their own material rather than everyone's average. It is a personal tool by Ricardo Remédio, used in the author's real composing sessions. The public repository ships without his catalogue, notes, judgments or reference library; see What is personal.

Learning from your catalogue is symbolic (MIDI notes, not audio). The optional listening features do handle audio and send it to an outside model; see What leaves your machine.

How it works

Your AI client (Claude Code, Codex, Cursor, Gemini CLI…) is the composer. It talks to two MCP servers: this one, which holds your catalogue and the composing tools, and Producer Pal, which drives Ableton Live.

flowchart LR
    sets["Your past Live Sets (.als)"]
    catalogue[("Your catalogue<br/>style profile, phrase index")]
    client["AI client<br/>(the composer)"]
    sidekick["composing-sidekick<br/>MCP server"]
    pp["Producer Pal<br/>MCP + Live device"]
    live["Ableton Live<br/>your open Set"]
    ledger[("Session ledger<br/>every suggestion, KEEP / DROP")]

    sets -- "pipeline, run once (ingest + analyze)" --> catalogue
    client -- "tool calls" --> sidekick
    sidekick -- "search, score" --> catalogue
    sidekick -- "read the Set, write SIDEKICK tracks, read back" --> pp
    pp <--> live
    sidekick -- "stamp each write" --> ledger

A session runs as a loop. The score that matters is survival: how many of the suggestions you judged are still in the Set at the end of the night.

flowchart LR
    hear["hear_the_room<br/>tempo, key, what is playing"]
    answer["answer_my_loop<br/>an answer from your own phrases"]
    write["write to SIDEKICK JAM<br/>read back, stamped in the ledger"]
    judge["you listen<br/>judge_answer: KEEP or DROP"]
    close["close_session<br/>which suggestions survived?"]

    hear --> answer --> write --> judge
    judge -- "next idea" --> hear
    judge -. "end of the night" .-> close

answer_my_loop works in "gears", from plain recall up to composing: 1 returns your nearest phrase as it is, 2 (the default) transposes and voice-leads it to fit your loop, 3 and 4 develop or extend it, and 5 and 6 let the AI compose under rules checked by the style critics. KEEP adds the source phrase to your "gold" favourites; DROP stops it, and phrases nearly the same, being served again (clear_rejects undoes that).

Related MCP server: AbletonMCP

What I decided vs what the agents wrote

• The idea, the rules and the judging are mine. I came up with the tool and keep refining it: what it's for and the rules it follows. My own material comes first and the generator is only a critic; automatic writes go only to tracks named SIDEKICK … and are read back; a night is scored by what survives, not by KEEP clicks. I judge every musical result myself.

• AI coding agents wrote most of the code and the tests, from my plans, and I reviewed their work. The plans, reviews and agent runbooks from the build are in docs/history/.

Core tools

The server has 77 tools. Profiles (SIDEKICK_MCP_PROFILE, see below) limit which ones the client sees, so it chooses among fewer: the default compose profile has 63, and a normal session uses a handful. Every tool: config/claude_mcp.md.

What it does

• Learns from your catalogue. ingest/ reads every MIDI clip in your Live Sets (arrangement, session and take lanes, and it knows which clips are drums). analyze/ turns them into a phrase index and statistics about your melodies, harmony, song structure, motifs and playing feel, under store/.

• Composes from your material first. answer_my_loop, find_phrases, develop_my_loop, grow_my_loop: call-and-response, variation and growth from the composer's own phrases. A seeded note generator exists, but only as a critic that scores ideas (score_phrase, critique, and the internal validate_composed check that write_composed_to_jam runs).

• Hears the room. hear_the_room reads the open Set through Producer Pal (tempo, key, playing clips, what role each plays). Audio clips are reported by name only, as named but unheard. An opt-in session ears pass (inventory_session_audio → listen_to_takes) can send takes to an outside listener model.

• Writes safely. These tools write automatically only to tracks named SIDEKICK JAM and SIDEKICK BOOTH. Every write is read back and checked; anything else is a preview or needs your confirmation.

• Closes the judgment loop. Every JAM write is stamped in a session ledger; the composer judges KEEP/DROP; close_session checks the Set and scores session survival (suggestions still in the Set at the end of the night), the project's real measure of success.

• Helps finish songs. whats_missing reads a project and says what the song still lacks; lay_the_song maps one scene into a full-song layout with locators, in proportions measured from the catalogue; three_endings offers three ways to end from the loop's last bar; bury_or_marry deals one of your saved ideas to keep or retire; one_hour_close turns a project's top findings into short finishing tasks for one hour that ends in a bounce.

• Uses references. analyze_audio_ref files a local file or URL as a listening card plus a MIDI sketch (Basic Pitch); find_audio_refs / apply_audio_ref / beat_from_ref let a reference steer which of the composer's own phrases and drum clips are used.

• Offline mix adviser (separate, in the debug profile): measurements on exported stems, at most three small gain changes, a loudness-matched blind A/B, and its own ledger.

Requirements

Install

git clone <this repo> composing-sidekick && cd composing-sidekick
python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt        # core: MCP server, pipeline, composing tools
.venv/bin/pip install -r requirements-test.txt   # what the test suite imports (CI installs this)
.venv/bin/python -m unittest discover -s tests -t .   # works on a clean clone (skips what needs your catalogue)

Optional, for audio references (analyze_audio_ref, Basic Pitch, YouTube fetch):

.venv/bin/pip install -r requirements-audio.txt
# basic-pitch pulls a TensorFlow stack on macOS that the code never imports (it runs on ONNX).
# Removing it saves space; afterwards `pip check` warns about basic-pitch, which is expected.
.venv/bin/pip uninstall -y tensorflow tensorflow-macos keras tensorboard

The server selftest (mcp_server/server.py --selftest, code in mcp_server/selftest.py) reads the generated style profile, so run it after Build your catalogue below. On a clean clone it stops with one line naming the missing files.

Build your catalogue

The store is empty on a clean clone. By default the scanner reads every Live Set under ~/Desktop, ~/Documents and ~/Music. SIDEKICK_CATALOG_ROOTS (separated by :) adds folders to those three, for example an external drive; only --roots replaces them, so use --roots to limit the scan to the folders you choose. Then run the pipeline in this order (each stage reads the previous one's artifacts):

export SIDEKICK_CATALOG_ROOTS="/Volumes/Drive/Ableton:/path/to/other/projects"   # adds to the defaults
python3 ingest/scan_catalog.py                # → store/catalog_manifest.json
# or scan only these folders: python3 ingest/scan_catalog.py --roots ~/Music/Ableton /path/to/projects
python3 ingest/materialize.py                 # → store/projects/*.json (one parse of the drive)
.venv/bin/python analyze/harmony.py           # music21 → store/harmony.json
python3 analyze/structure.py                  # → store/structure.json
.venv/bin/python analyze/style_profile.py     # AFTER harmony + structure → style_profile.json + STYLE.md
python3 analyze/build_index.py                # → store/phrase_index.json
python3 analyze/motifs.py                     # AFTER build_index + style_profile
.venv/bin/python tools/build_directions.py    # direction axes (reads the profile)
# optional digs
.venv/bin/python tools/harvest_arrangement_dna.py && .venv/bin/python tools/build_phrase_curves.py
.venv/bin/python tools/build_groove_templates.py

tools/rebuild_corpus.py --commit runs scan → materialize → analyze with fail-closed gates (every previously scanned root still mounted, project counts within 5 %) and restores the previous artifacts if a stage fails. Pack and demo Sets you do not want learned from go in store/not_his_projects.json ({"blocked_projects": ["<project name>", …]}, matched case-insensitively); every analyzer honours it.

store/NOTES.md is yours: hand-written notes that survive every rebuild and are read by the assistants and by style_profile.py.

MCP setup

Start from the example: cp .mcp.example.json .mcp.json (.mcp.json is gitignored), then fill in the absolute path of your clone (MCP hosts do not expand ~). For another client, put the same block in its MCP config. The server registers as style-sidekick, its original name, so its tools show up as mcp__style-sidekick__*.

{
  "mcpServers": {
    "style-sidekick": {
      "command": "<repo>/.venv/bin/python",
      "args": ["<repo>/mcp_server/server.py"],
      "env": { "SIDEKICK_MCP_PROFILE": "compose" }
    },
    "producer-pal": {
      "command": "npx",
      "args": ["-y", "producer-pal@2.3.0", "-f", "json", "-n", "midi-json"]
    }
  }
}

Install the Producer Pal device on a MIDI track in your Set (producer-pal.org/installation), restart the client, and start with a read-only ppal-connect: the sidekick checks the reported producerPalVersion against its captured builds before any write. The Live endpoint is http://localhost:3350/mcp (PRODUCER_PAL_URL overrides it).

Ask before raw Producer Pal writes

The sidekick's own tools only write to SIDEKICK … tracks, but the same client also has the full producer-pal server, which can create, change or delete anything in the Set. In Claude Code, copy docs/examples/claude-settings.json to .claude/settings.json (or merge its permissions.ask list into yours). It makes Claude ask you before every Producer Pal create, update, duplicate or delete call, and before ppal-live-api. Reads and playback stay unprompted.

Profiles (SIDEKICK_MCP_PROFILE)

Tool reference, ears workflow, judgment ledger and audio-reference workflow: config/claude_mcp.md. How a session runs: docs/CURSOR_CHAT_PLAYBOOK.md.

Tests

.venv/bin/python -m unittest discover -s tests -t .    # stdlib unittest; pytest is not installed
.venv/bin/python mcp_server/server.py --selftest        # offline; needs the built catalogue; must end green

The suite never contacts Live: tests/_guard.py fails any test that reaches the Producer Pal port, and the selftest snapshots the store before running and refuses to leave it changed. Tests that need catalogue-generated artifacts, ffmpeg, or a Producer Pal capture that was never made skip with a reason; on a clean clone expect a few hundred skips, zero failures.

Producer Pal behaviour is tested against reply envelopes captured from the real device under tests/fixtures/pp/ (read, mutate, legacy and version-specific runs per build): the shapes, keys, orderings, warning and error texts are the device's. The music inside them is synthetic: names are invented and the piece replay's notes are newly drawn over the captured arrangement skeleton (tests/fixtures/pp/README.md). The fakes in tests/pp_fakes.py validate every call against that build's tool schemas. Adding a build means capturing it on a disposable scratch Set first — see docs/PRODUCER_PAL_VERSIONS.md.

What leaves your machine

Learning from your catalogue is symbolic and local: the pipeline reads the MIDI inside your .als files and writes its results under store/. Three optional features work with audio, and each one sends audio to Google's Gemini model through the agy CLI:

• Audio references (analyze_audio_ref, analyze_audio_refs_batch). Given a YouTube URL, the tool downloads the audio with yt-dlp. The describe mode (on by default) then sends that audio, or a local file you name, to Gemini for a listening card. The midi (Basic Pitch) and features modes run locally. Only use recordings you have the right to use.

• Session ears (listen_to_takes, opt-in). Sends audio takes from your open Set, one at a time and cut to at most 45 seconds, to Gemini.

• Mix witness (analyze_mix_pass(listener=True), off by default). Sends the exported master and group stems to Gemini.

Without agy installed, no audio is sent and the rest keeps working. Like any MCP server, the sidekick's tool results (notes, phrases, style facts) also go to the LLM client you connect, so your client's provider sees them.

What is personal and not in the repo

Everything the pipeline derives from a catalogue is regenerated locally and gitignored: corpus/, store/projects/, store/phrase_index.json, style_profile.json, harmony.json, structure.json, motifs.json, taste_model.json, automation.json, phrase_curves.json, directions.json, store/audio_refs/, store/session_ears/, store/exports/, tools/ideas/ experiment data, .scratch/.

Files that hold a composer's words and judgments (store/NOTES.md, STYLE.md, ratings.json, session_ledger.json, rituals.json, pieces.json, piece_records.json, piece_decisions.json, booth and audition votes) are created on your machine the first time you use the matching tool. The public repository contains none of the author's. The musical-knowledge skills under .claude/skills/ ship as templates for the same reason: the idiom, theory and listening notes they held were his.

None of the author's words, melodies, rhythms or measurements ship. Where the code needs a value or a fixture that used to come from his catalogue, it now has a neutral or synthetic stand-in (and, for calibration, an optional local override). The one structural remainder is named below:

• Producer Pal fixtures (tests/fixtures/pp/**): captured reply envelopes with invented names and newly drawn notes; no note matches one in its corresponding original clip. The piece replay keeps the captured arrangement skeleton (clip positions, lengths, loop windows) and its per-bar harmonic outline; see tests/fixtures/pp/README.md.

• Engine calibration (mcp_server/idiom_cells.py): neutral defaults (common four-note cells, a moderate human-ish feel), tagged evidence: "default". Put values measured on your own catalogue in the gitignored store/calibration.json (keys cells, feel) to override them. Role thresholds (analyze/roles.py) are plain conventional register and texture boundaries, not measured on anyone.

• Rhythm library: none ships. If you keep one, it lives in the gitignored store/approved_rhythms.json with its MIDI under assets/approved_rhythms/; the loader and get_style() tolerate its absence.

• Audio-reference fixtures (tests/fixtures/audio_ref/): generated by make_synthetic.py in that folder — seeded plateau energy curves and invented cards designed to trigger the detectors the tests assert, measured on no recording.

Status (2026-10-14)

A working prototype, used in the author's real composing sessions. The current product arc is durable KEEP: fewer features, more suggestions that survive the session. Parked by design: SOMBRA, a full Max for Live pedalboard control surface (its transport and OSC protocol exist, docs/SOMBRA_DEVICE.md), a core/server split, stem separation, new note generators, automatic mixing. Ears-by-default is gated on a pre-registered blind round (docs/EARS_PROMOTION.md).

Licence

MIT — see LICENSE. Copyright (c) 2026 Ricardo Remédio.

Documentation map