<img src="docs/images/logo.png" width="50%" align="right" alt="bespoke-mcp">

# bespoke-mcp

An [MCP](https://modelcontextprotocol.io) server that lets an LLM drive a live
[Bespoke Synth](https://www.bespokesynth.com/) window: build and rewire the module graph, set and
modulate controls (including the right-click LFO surface), sequence notes, control playback, and —
crucially — *hear the result* via numeric audio analysis, rendered spectrogram/waveform images, and a
symbolic note-event log fed back to the model.

## Install

**[INSTALL.md](INSTALL.md)** is the setup: what has to be on the machine (git, CMake, a C++
compiler, Python 3.11–3.14, `uv`, ffmpeg, optionally Docker), then Windows and Linux
instructions per component. Once the submodules are cloned and the fork is built, `./run.sh`
brings up a window with the bridge open and every harness below sees the servers.

## Every harness, no configuration

The four servers — `bespoke-live`, `bespoke-offline`, `score` and `overtone` — are registered **in the repo**,
one config file per harness, so opening this directory in any of them is the whole setup. None of
the files names a path: `uv run` resolves the project from the working directory, which is this
checkout.

| Harness | File | Notes |
|---|---|---|
| [Claude Code](https://claude.com/claude-code) | [.mcp.json](.mcp.json) + [.claude/settings.json](.claude/settings.json) | `enableAllProjectMcpServers` approves all four on open, so `/mcp` lists them with nothing to click |
| [VS Code / Copilot](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) | [.vscode/mcp.json](.vscode/mcp.json) | VS Code asks once before starting a workspace server |
| [Cursor](https://cursor.com/docs/context/mcp) | [.cursor/mcp.json](.cursor/mcp.json) | |
| [OpenCode](https://opencode.ai/docs/mcp-servers/) | [opencode.json](opencode.json) | |
| [Codex CLI](https://developers.openai.com/codex/mcp) | [.codex/config.toml](.codex/config.toml) | project-scoped config needs the directory trusted |
| [Gemini CLI](https://geminicli.com/docs/tools/mcp-server/) | [.gemini/settings.json](.gemini/settings.json) | |
| [Zed](https://zed.dev/docs/ai/mcp) | [.zed/settings.json](.zed/settings.json) | Zed calls them `context_servers` |
| [Antigravity](https://antigravity.google/docs/mcp), [Kimi Code](https://moonshotai.github.io/kimi-cli/en/customization/mcp.html), [Windsurf](https://docs.windsurf.com/windsurf/cascade/mcp) | *per user* | these keep one central config in your home directory and will not take a relative path, so it cannot be committed. `uv run python tools/install_agent_config.py --install antigravity kimi windsurf` writes it, merging into whatever is already there |

**Two harnesses at once.** Every line above is `uv run --frozen --no-sync python -m bespoke_mcp`
and not `uv run bespoke-mcp`, which is what makes Claude Code and Antigravity — or any other two —
able to hold these four servers open on the same checkout at the same time. A plain `uv run
<script>` re-installs the project before it launches anything, and on Windows re-installing means
deleting `.venv/Scripts/bespoke-mcp.exe`, which the OS refuses while another client is *running*
that file:

```
bespoke-live: error: failed to remove file `…/.venv/Scripts/bespoke-mcp.exe`:
The process cannot access the file because it is being used by another process. (os error 32)
```

It is not really a two-harness problem — one harness starting four servers at once races with
itself the same way, and any edit to `pyproject.toml` re-arms it, because that is what marks the
install stale. Two flags and a module path defuse all of it: `--no-sync` and `--frozen` mean
nothing writes the environment or the lockfile on startup, and `python -m` means no client ever
holds the `.exe` open, so `uv sync` works with every harness still running. The trade is that the
environment is now built *only* by an explicit `uv sync` — the project is installed editable, so
repo edits are live regardless; it is dependency changes that need the sync. A clone that skips it
gets `No module named bespoke_mcp` rather than a silently stale server.

The **skill library** is [.agents/skills/](.agents/skills/README.md) — thirty-five skills, and
that is the only place it lives. There is nothing to install and no per-harness copy: the path is
named here and in [AGENTS.md](AGENTS.md), which every harness above reads as the project brief.
(There used to be `.claude/skills` and `.opencode/skills` symlinked to it. A Windows clone turns a
committed symlink into a seventeen-byte text file, so the fallback was a *copy* that went stale
after every skill edit — a second source of truth for no extra reach.)

Prose context is [AGENTS.md](AGENTS.md), which Claude Code, Codex, opencode, Gemini CLI, Cursor
and Zed all read as the project brief.

## Architecture

```
Claude ──stdio──▶  Python MCP server  ──TCP NDJSON JSON-RPC──▶  BespokeSynth.exe (fork)
                   (mcp/src/bespoke_mcp)                         └─ McpBridge (C++ singleton)
                   numpy DSP · matplotlib plots                     drained on the main thread
                   ASCII piano roll · module catalog                in ModularSynth::Poll()
```

- **C++ bridge** (`BespokeSynth/Source/Mcp*.{h,cpp}`) — an always-on singleton registered via
  `ModularSynth::AddExtraPoller`. Socket threads parse newline-delimited JSON-RPC 2.0 on
  `127.0.0.1:5309` (pref `mcp_port`, env `BESPOKE_MCP_PORT`, CLI `-o mcp_port N`) and queue requests;
  all graph mutation executes on the message thread with the same locks the UI uses. Structured errors
  carry `did_you_mean` / `valid_targets` payloads so a model self-corrects on the next turn.
- **Python MCP server** (`mcp/src/bespoke_mcp`) — exposes the bridge as MCP tools, does the audio
  analysis in numpy (dBFS, spectral centroid/flatness/rolloff, band energies, pitch, onsets,
  stereo) with `pyloudnorm` for BS.1770 loudness and `audioflux` for the per-stem spectral
  battery, renders PNG plots for vision, and formats note/pulse events as an ASCII piano roll.
  **It registers twice**, as two servers over one codebase differing only in a launch flag:

  | Server | Launch | Lifecycle | Extra tools |
  |---|---|---|---|
  | `bespoke-live` | `bespoke-mcp --mode live` | attaches to the window `./run.sh` started; refuses to spawn | — |
  | `bespoke-offline` | `bespoke-mcp --mode offline` | owns a private headless *stepped* synth on a private port, torn down on exit | `bespoke_project_open` / `_save`, `bespoke_render`, `bespoke_render_status` |

  Both are in [.mcp.json](.mcp.json), so a checkout has them without anyone running
  `claude mcp add`. Mode is bound at process start and cannot move: the two differ in process
  lifecycle, which is a launch-time property, and binding it there means the unchosen mode's
  tools do not exist in that server rather than erroring when called. Every dict-shaped reply
  carries `server_mode`, so a transcript says which one produced it.
- **`BespokeSynth/`** — git submodule pointing at our fork (`Crack-Pantelimon/BespokeSynth`).
  Upstream file touches are kept minimal and listed in `UPSTREAM.md`, which is swept mechanically
  against `git diff --name-status`.
- **`data-packs/`** — git submodule (`Crack-Pantelimon/bespoke-mcp-data-pack`) holding the samples
  and plugin bundles themselves, 5.4 GB of them, versioned rather than re-downloaded.

## Demos

`data-packs/demos/projects/` holds the **six active demos**, each built end-to-end through these
MCP servers — spawned, wired, sequenced, modulated, laid out, played and recorded by driving the
tools, with no hand editing. (Thirty-nine older ones are frozen under `demos/historical/`; see
below.) A demo ships as:

- `<name>.bsk` — the savestate (structure **and** live control values); open it in Bespoke
- `<name>.mp3` — the render off the master bus. **mp3 is what gets committed**; the `.wav` beside
  it stays on disk and is gitignored (`demos/**/*.wav`), because a wav is roughly six times the
  size of a `-q:a 0` VBR mp3 of the same audio: **654 MB against 103 MB across the demos**, and
  the render was already 97.7 % of a demo project's bytes
- `<name>.json` — the analysis, the verdict, the per-section table and an ASCII piano roll
- `<name>_waveform.png`, `<name>_spectrogram.png`
- `bespoke/instruments/<track>/` where a piece has been realised per-instrument: one stem per
  part (mp3, QA images, `analysis.json`). A realised score gets all of them out of the mix's own
  render through per-module recording taps, so the stems are sample-aligned with the master;
  `bespoke_render_stem` is the one-part-at-a-time version for building a patch by hand

Rebuild them all with `uv run python tools/build_demos.py`, or one with
`uv run python tools/build_demos.py acid_303`. Drive a private headless instance instead of
whatever is on the default port with
`BESPOKE_MCP_PORT=5310 BESPOKE_DATA_DIR=… uv run python tools/build_demos.py <style>`
(`BESPOKE_DEMO_DIR` redirects the output too).

**Long form.** Every one is a full piece rather than a loop: named sections,
per-section instrumentation, and filter/level moves scheduled on the bar with
`control.set {at_measure_time}`. A style declares its own length with
`p.arrange(("intro", 4), ("groove", 8), …)` and the builder renders and captures all of it,
chunking the capture past the bridge's 30 s-per-call limit. Such a style is additionally checked
for *being a piece of music* — length, per-section loudness/brightness contrast, how many bars
differ from every earlier bar, and how many instruments actually play. See
[.agents/skills/bespoke-arranging/SKILL.md](.agents/skills/bespoke-arranging/SKILL.md) for the
helper table and [docs/Composition_Standard.md](docs/Composition_Standard.md) for
the composition method the newest ones were written against.

The five genre skills — [orchestral](.agents/skills/bespoke-orchestral/SKILL.md),
[jazz](.agents/skills/bespoke-jazz/SKILL.md),
[electronic](.agents/skills/bespoke-electronic/SKILL.md),
[rock/metal](.agents/skills/bespoke-rock-metal/SKILL.md) and
[hip-hop](.agents/skills/bespoke-hiphop/SKILL.md) — each map a genre's harmonic, rhythmic and
mixing vocabulary onto the samples, plugins and presets that are actually on disk. Their worked
examples are the `genre_*` demos, now under `demos/historical/`, so read them for the technique
and expect their measurements to move when they are rebuilt.

Six **instrument-family** skills are the second axis: a genre skill says which instrument and why,
and [drums](.agents/skills/bespoke-drums/SKILL.md), [bass](.agents/skills/bespoke-bass/SKILL.md),
[keys](.agents/skills/bespoke-keys/SKILL.md), [strings](.agents/skills/bespoke-strings/SKILL.md),
[brass and winds](.agents/skills/bespoke-brass-winds/SKILL.md) and
[guitar](.agents/skills/bespoke-guitar/SKILL.md) say how to play and produce it — the inventory
named on disk, the articulations that exist, a measured level for every map, and the honest
limits.

Four **component and production** skills are the third axis:
[synths](.agents/skills/bespoke-synths/SKILL.md) (Surge XT's 637 factory patches and the rest of
the hosted fleet, the stock `oscillator` as a full subtractive synth, and recipes for pad, lead,
pluck, stab, riser and drone), [sampling](.agents/skills/bespoke-sampling/SKILL.md) (the eight
drop-target modules, the cue table and chop-and-flip, granular, and sfizz `.sfz` state injection),
[effects](.agents/skills/bespoke-effects/SKILL.md) (the 18 stock effects and ~30 hosted bundles by
category, insert versus master versus send) and
[mixing/mastering](.agents/skills/bespoke-mixing-mastering/SKILL.md) (gain staging against 17–36 dB
voice spreads, width, mono compatibility, the master chain, and targets computed from all 43
committed demos).

Seven **score** skills are the fourth axis, for composing rather than patching:
[songwriting](.agents/skills/score-songwriting/SKILL.md) is the whole loop in one file, with
planning, constraints, writing, dynamics and checking as its steps — and
[handoff](.agents/skills/score-handoff/SKILL.md), which is the flagship for Bespoke work that
*starts* from a composed score: the tempo map and the measure-time queue, one instrument at a
time through `bespoke_render_stem`, seating and depth, loudness-matched stems, and the mandate
that nothing in the patch stays static.

**The reading order is the point of the library**, and
[.agents/skills/README.md](.agents/skills/README.md) is its entry point: if a score already
exists, start at handoff; otherwise read the genre skill, then the family skill for every
instrument that genre recommends, then the component skills the work touches — and only then
start editing a track.

| Style | BPM | Modules | Cables | What it is |
|---|---|---|---|---|
| `score_wide_water` | 96→112 | 23 | 19 | **40 bars / 105 s — the only one here that was *composed*, not patched.** Written note by note in `score-mcp` under six declared constraints, then handed to Bespoke through `bespoke_load_score` and realised. D dorian at four distances: a cello line (not a pad), a flute that enters on the sixth and never touches the tonic, horns that arrive exactly once as a swell, three timpani strokes in the whole piece. Two writes were **rejected by its own rules** and both replacements were better than what they replaced. Score, engraving, stems, text mirror and hand-off bundle all committed beside the render. Journal: `data-packs/demos/projects/score_wide_water/`. |
| `prog_death_metal` | 168 | 33 | 27 | **64 bars / 96 s**: progressive death metal composed in score-mcp — a four-note cell seven sixteenths long argued against a four-beat bar, through blast, unison stops with a bar of silence, half-time, five-sixteenth groups, a quartal clean interlude, a harmonic-minor solo, then the riff a tritone up on A♭ that never comes home. No vertical major thirds. Six stems, each measured; the bass the old version shipped broken produced 6.6 attacks per written note. Journal: [compositional_journal.md](data-packs/demos/projects/prog_death_metal/compositional_journal.md). |
| `manele` | 104 | 51 | 49 | **40 bars / 99 s**: a modern manea composed in score-mcp — the düyek grid with nothing on beat 2, a Hicaz tetrachord actually in tune (F +13 ¢, G# −15 ¢ through note-gated pitchbenders, measured back off the stem to a third of a cent), an Andalusian refrain, a clarinet solo behind the beat, and a move to the Hicaz on A that never returns. Journal: [compositional_journal.md](data-packs/demos/projects/manele/compositional_journal.md). |
| `acid_303` | 130 | 41 | 37 | **36 bars / 67 s**: two JC303s — a driven main line and an octave-up answer — with cutoff, resonance, envmod and decay ridden per section. |
| `breakbeat` | 136 | 47 | 43 | **32 bars / 57 s**: big beat — programmed kit under a re-cut amen, hoover and rave one-shots, Wolf Shaper fuzz bass. |

**The other 39 demos are frozen, not deleted.** They live under `data-packs/demos/historical/<name>/` with their renders, savestates, analyses and journals intact, and they are out of the project index and out of `styles.py`. They predate several engine fixes, so re-rendering them would only make them wrong more recently; they want re-researching and re-composing through the current pipeline, which is a batch of its own (see [TODO.md](TODO.md)). Their history is untouched, so `git log` still explains every one of them.

All five build cleanly: audible, no NaN/Inf, no clipping, and zero overlapping modules after
auto-layout.

## Deterministic rendering

The fork ships an `OfflineAudioIODevice` — a real JUCE device type named `Offline`, so
`-o devicetype Offline` selects it through Bespoke's normal audio-init path with no sound card
involved. It renders on its own thread, **stepped**: nothing renders until `offline.render`
asks for buffers, one buffer at a time. Wall-clock time stops being an input to the result, so
the same patch renders the same event log every time.

That is the only offline mode. A free-running one existed (and an ALSA-null-sink trick before
it) until the score/Bespoke split deleted both: audio raced 30–600× ahead of the clock, which
made every `bespoke_wait` overshoot by two orders of magnitude and no render reproducible.

Drive it with `bespoke_wait(measures=...)`, never a sleep — the tool asks the synth what it is
and either renders exactly the buffers needed or polls the transport. `tests/golden/` uses this
to assert **three** patches note-for-note (plus a determinism test that renders one of them
twice and diffs the event logs, ~65 s for the suite); see
[tests/golden/README.md](tests/golden/README.md) for what had to be pinned (seed, scale, tempo,
drum kits) and why, and `tests/golden/retired/` for the three that were recorded and dropped.

Control writes can be scheduled on the musical grid rather than executed now:
`bespoke_set_control(..., at_measure_time=17.0)` queues the write inside the bridge and fires it
on the audio-buffer boundary that owns that measure — between the buffers of a stepped render as
well, so a scripted mix move lands on the same sample in every run. The queue drains against
**measure** time rather than milliseconds, so automating `transport~tempo` mid-piece no longer
drags everything queued behind it, and `offline.render {"measures": N}` renders until N measures
have actually elapsed rather than multiplying by the current bar length. `state.clear` cancels
the queue; a transport reset deliberately does not, because a scheduled write is a position on
the musical timeline and a reset moves the timeline under it.

## Judging a render

Every render is measured before it is called done. `bespoke_render` returns a **quality verdict**
next to the analysis — integrated LUFS and LRA (BS.1770), a short-term LUFS timeline, true peak
from a 4×-oversampled signal, clipping runs, DC, crest, spectral statistics — against the
streaming targets (−14 ± 1 LUFS-I, ≤ −1 dBTP, LRA ≥ 4). `mcp/src/bespoke_mcp/analysis/quality.py`.

`bespoke_render_stem(track)` renders **one instrument alone** into
`bespoke/instruments/<track>/` and analyses it **against that part's own notes from the score**,
which is the only way to ask questions a mixed render cannot answer: does the audio move when
this part plays, does it produce one onset per note or nine, does the pitch match what was
written, is the timbre moving at all. Flags are `SILENT_WHILE_MIDI_ACTIVE`, `CLICKY`,
`PITCH_MISMATCH`, `NOISE_LIKE`, `STATIC_TIMBRE`, `CLIPPING`, `DC_OFFSET`, `HOT_STEM`,
`DEAD_STEM`. It exists because `prog_death_metal` shipped a bass preset that renders as repeated
beats: nothing in the pipeline had ever rendered a single instrument on its own.

`bespoke_render(semantic_tags=True)` additionally runs Essentia's genre / mood / danceability
models through `./run-docker-essentia.sh` (a quarantined Linux/CPU-only image, so the 292 MB
wheel never enters the main venv and the same path works from Windows; weights vendored in
`data-packs/models/essentia/` under **CC BY-NC-SA 4.0**, downloaded and SHA256-pinned by
`tools/fetch_models.py`). The tags are noisy and advisory — never a check.

## score-mcp — the symbolic half

`score/src/score_mcp/` is a second MCP server (`uv run score-mcp`, registered in
[.mcp.json](.mcp.json)) that composes rather than patches. One SMF Format 1 file per piece, one
track per instrument, **track 0 the Conductor Notes track** — the tempo and meter map, the
section markers, and a line of narrative every few bars saying what is happening and what the
mix should do about it. Everything a musician or a DAW should see lives in the `.mid`; the
sidecar holds only what no downstream consumer would honour.

What it is built around:

- **One write tool, one revision per edit.** `score_edit(edits=[…])` stages notes, comments,
  conductor notes, CC, articulations, dynamics, copies and transforms together and commits them
  as one revision and one engraving. The six separate write verbs it replaced meant six commits
  over what was conceptually one edit — and a blocking rule about comments that fired *between*
  them, turning documentation into a toll gate. Prose about a region now travels with the region.
- **Rejection is per item.** An item that would create a blocking violation is dropped, with the
  violating note indices named; the rest of the call still lands. A rejection costs the corrected
  item, not the whole payload.
- **Read before you write, enforced.** Every read reports a revision; every edit takes
  `expect_revision` and is refused on a mismatch, with the journal entries that landed in
  between. An agent cannot write a region it has not read at its current state, and a rejected
  edit does not advance the revision.
- **Caps that bias small.** 32 bars × 8 tracks hard, per item.
- **Repetition without re-emitting notes.** An item's `copy` and `transform` (transpose, invert,
  retrograde, stretch, quantize, humanize, velocity_curve) are how a hundred-bar piece is built
  out of eight-bar decisions.
- **The idiom, measured.** `score_reference` imports real MIDI through the verification battery
  (magic bytes, header sanity, full parse, duration and track count against what the page
  claimed, copyright metas surfaced, performance-vs-engraving triage, suspect-tempo-tag flag)
  into a per-project `references/`. `score_analyze(refs=[…])` runs six metric families over them
  **chaptered** — density, chord size, pitch-class saturation, spiral-array tension, syncopation
  and IOI entropy, microtiming — and `score_reference_compare` puts the piece's own sections
  beside the reference chapters they were calibrated on. The same code measures both sides,
  which is the only way the comparison means anything.
- **Structure is one splice.** `score_insert_bars` moves notes, markers, comments, tempo and
  meter together — which is why comments live in the file and can never point at a stale bar.
- **Every write engraves, and the prose is on the page.** `score/<name>.pdf` and
  `score/<name>.jpg` land next to the `.mid` via music21 → verovio → svglib → pypdfium2: five pip
  wheels, no LilyPond, no MuseScore. Conductor notes and per-track comments are engraved at their
  bars as numbered, truncated markings, with an appendix at the back of the PDF listing every one
  in full. The appendix is PDF-only — the JPG is the strip of pages an agent looks at, and pages
  of prose in it would push the music off the screen.
- **Tempo is a curve on the barline.** `score_setup(tempo=[…])` snaps to whole bars unless
  `allow_mid_bar=True`, and a ramp emits one event per **bar** on a smoothstep S-curve — the
  gesture a conductor actually makes. A twelve-bar accelerando is thirteen events rather than
  fifty, which is what keeps a piece inside the 4–20 tempo regions it should have; and
  `bespoke_load_score(tempo_map=True)` schedules the whole map, so the ramp is in the render.
- **Complexity floors, which never block.** `score_check` returns twelve measured floors —
  duration variety, rhythmic strata, harmonic density and motion, velocity variety, CC motion,
  role coverage, density curve, section contrast, low interval limits, anti-loop, tempo grid —
  graded `note` / `warning` / `severe` by how far under the floor the measurement sits, each
  naming the next pass to make. There is no waiver and no genre exemption: a first pass that
  states the material simply is good work, and the floors are the composer's to-do list rather
  than a verdict.
- **Constraints reject the item, not the file.** Sixteen types, each generalising an assertion
  a demo already made in Python (`cue_dread`'s pitch-class plan, `prog_death_metal`'s no major
  thirds, `salsa_dura`'s bass off the downbeat), plus an AST-sandboxed `expression` for the
  one-offs where the artistic value concentrates — and `chord_tone_agreement`, the positive,
  octave-aware form of a unity rule that had failed as a pitch-class interval ban and was
  silently dropped. A `block` constraint refuses an item that would create a violation — the
  score on disk is untouched and you fix the notes.
- **The analysis battery is symbolic.** Harmony (chord timeline, K-S key, dissonance curve),
  energy (velocity and density against a genre template), repetition (distinct bars, per-track
  loopiness, an SSM PNG, Foote novelty against the *declared* boundaries), voice leading and
  role coverage — all arithmetic over the notes, because the harmony of a piece is a property
  of its notes and inferring it from a render is measuring the shadow.
- **A `dynamics` block writes hairpins as sound.** Velocity for anything struck; CC1 where
  a sampled library crossfades to a different *recording* (turning up a `p` sample never sounds
  like `f`); CC74 where a synth crescendo is the filter opening; `expression=True` for CC11 as a
  **phrase arc**, which is not the verbatim copy of CC1 it used to be — that duplication was 964
  of the 1 124 lines in one 40-bar piece's text mirror, saying nothing twice. The `dyn:` marking
  is written in the same call, so the word and the notes under it cannot drift apart.
- **Every write answers globally.** The feedback bundle carries the constraint *delta* (broke /
  fixed), what the edit did to the energy, repetition and register curves in four sentences,
  and the failing lints — so "what did that do to the whole track" never costs another call.

## Editing a patch that already exists

A patch is a graph, and until S2 the only well-supported edit to one was "build another".
Six tools change that, all of them one command under one lock so the audio thread never
observes a half-rewired graph:

| Tool | Change |
|---|---|
| `bespoke_replace_module` | swap the node, keep the wiring, carry every same-named control across (`map_controls` renames or drops) |
| `bespoke_insert_between` | spawn a module inside an existing cable; restores the original cable if the new type cannot carry what it carries |
| `bespoke_delete_module(heal="reconnect")` | remove a node and re-point its upstreams at its downstreams, reporting anything left unfed |
| `bespoke_rewire` | a batch of cable moves applied together |
| `bespoke_reset_module_state` | clear the state no control shows — a notesequencer's random constructor fill, a notecanvas's notes, every grid |
| `bespoke_graph_diff` | two exported patches subtracted: modules added/removed/retyped, cables, and every control that drifted with its old and new value |

`bespoke_clear_all` refuses until you pass `confirm_destroys=N` matching the real module
count — the destructive path is the one that now costs a read.

## Arrangement

A loop is not a track. Four surfaces turn one into the other, all of them driving the same
methods the mouse drives:

| Tool group | Module | What it buys |
|---|---|---|
| `bespoke_add_snapshot_target` / `bespoke_store_snapshot` / `bespoke_recall_snapshot` / `bespoke_list_snapshots` | `snapshots` | scenes: capture every tracked control's value into a slot, recall it later (optionally blended over N ms) |
| `bespoke_songbuilder_*` | `songbuilder` | sections: named scenes with a value per target control, sequenced as `[{scene, bars}]`, played with `bespoke_songbuilder_play` |
| `bespoke_set_canvas_notes` / `bespoke_get_canvas_notes` / `bespoke_load_midi_file` | `notecanvas` | a piano roll: free positions, arbitrary lengths, overlapping notes, a velocity each — and chords, which a monophonic notesequencer silently collapses |
| `bespoke_set_euclidean`, `bespoke_set_steps` on `basslinesequencer`, `bespoke_get_sample_cues` / `bespoke_autoslice_sample` | `euclideansequencer`, `basslinesequencer`, `sampleplayer` | euclidean rings, 303 accent/slide patterns, and chop-and-flip sample slicing |

The canonical workflow — build the full-energy loop, store one snapshot per section, then
sequence the sections — is written up in
[.agents/skills/bespoke-arranging/SKILL.md](.agents/skills/bespoke-arranging/SKILL.md) and asserted end to end in
`tests/integration/test_arrangement.py`.

## Seeing and moving a patch

Three things an agent could not do before: watch a script, look at the UI, and carry a patch.

- **`bespoke_get_events`** now returns four kinds in one ordered stream — `note`, `pulse`,
  `script` (every `me.output()` line) and `error` (every exception a script raised), each tagged
  with the module that produced it and filterable with `kinds=[...]`. A generative script that
  degrades halfway through a render used to be invisible until somebody thought to poll it.
- **`bespoke_screenshot`** captures the synth's own framebuffer: the whole window, or one
  module's rectangle. That is the only way to read the displays that have no programmatic
  accessor — a notecanvas's notes, a waveform, a meter. It works headless, because llvmpipe's
  offscreen framebuffer is a real one.
- **`bespoke_export_patch` / `bespoke_import_patch`** write and rebuild the whole patch as JSON:
  modules, positions, cables, every control value, save-data, LFO and modulator attachments,
  sequencer grids, notecanvas notes, sampleplayer cues, effectchain contents, script source and
  hosted-plugin state blobs. Layout JSON carries none of that — a knob position exists nowhere
  but the binary `.bsk`. Import is deliberately not atomic: every step reports its own success,
  so one unloadable plugin leaves you the rest of the patch and a list of what to fix.

## Development

```sh
# build the fork and start it windowed with the bridge open (Windows/Git Bash, Linux, macOS)
./run.sh                                    # --restart / --stop / --no-build / --build-only
./run.sh --port 5310 -o audio_output_device none

# C++ (from BespokeSynth/): configure once, then
cmake --build ignore/build --config Release --target BespokeSynth

# C++ unit tests (fetches Catch2; not the GUI app, so it builds in seconds)
cmake -S . -B ignore/build-tests -DBESPOKE_BUILD_TESTS=ON
cmake --build ignore/build-tests --target bespoke-mcp-tests && ./ignore/build-tests/tests/bespoke-mcp-tests

# both submodules; data-packs is 5.4 GB of samples and plugin bundles, so this takes a while
git submodule update --init --recursive

# Python: uv manages everything from the repo root
uv sync
uv run pytest
uv run ruff check .

# the three MCP servers (registered in .mcp.json; this is how to drive one by hand)
uv run bespoke-mcp --mode live       # attach to the ./run.sh window
uv run bespoke-mcp --mode offline    # own a private headless stepped synth
uv run score-mcp                     # compose: notes, structure, engraving, hand-off

# headless: no window, no sound card, bridge open on 5309
tools/run_headless.sh                       # deterministic; nothing renders until asked

# these launch their own headless synth if you do not have one open
uv run pytest tests/integration -v
uv run pytest tests/golden -v
uv run python tools/update_goldens.py --check

# the plugin fleet: named parameters and the shipped preset bank
uv run python tools/verify_profiles.py      # 52 profiles, 730 indices, checked live
uv run python tools/build_presets.py --verify

# the sampled orchestra: parse + pitch-verify the packs, then write their .sfz key maps
uv run python tools/make_sfz.py --survey
uv run python tools/make_sfz.py --generate

# the vendored Essentia models, and the container that runs them
uv run python tools/fetch_models.py             # download + SHA256-pin into data-packs/models/
./run-docker-essentia.sh path/to/render.wav     # builds the image on first use

# the audio lab: separation and (later) transcription for reference audio
uv run python tools/fetch_models.py --set audio-lab   # gitignored weights, fetched per host
./run-docker-audio-lab.sh selftest                    # builds the image on first use
./run-docker-audio-lab.sh separate mix.wav --out /tmp/lab-out
./run-docker-audio-lab.sh triage mix.wav --out /tmp/lab-out       # which stages are worth running
./run-docker-audio-lab.sh destructure mix.wav --out /tmp/lab-out  # all of them, one container start

# Run the MCP server (spawns/attaches to BespokeSynth via <data>/mcp/instance.json)
uv run bespoke-mcp
```

**Dependencies.** `uv sync` installs everything: `mcp<2` (the 2.0 SDK dropped
`mcp.server.fastmcp`), `numpy`, `matplotlib`, `pillow`, `pydantic`; `pyloudnorm` and `audioflux`
for the quality battery, both pure/universal wheels so they work on Windows too; and `mido`,
`music21`, `verovio`, `svglib`, `pypdfium2` for score-mcp — five pip wheels rather than the
obvious LilyPond or MuseScore, neither of which is installable alongside the rest.

Two things are expected on `PATH` rather than in the venv: **ffmpeg** with `libmp3lame` (every
render transcodes to the mp3 that gets committed, and the audio reference battery decodes every
acquired file through it — **nothing in the venv opens an mp3, m4a or opus**, so `audio_reference`
without ffmpeg verifies magic bytes and stops there, saying so) and, only for `semantic_tags=True`
and the audio lab, **docker**. Essentia itself deliberately never enters the venv — the pinned wheel is
292 MB and Linux-only, so it lives in `docker/essentia/` and the same command works from Windows.
The same rule, one tier out, keeps torch, demucs, transformers and basic-pitch in
`docker/audio-lab/`: 2.26 GB of Linux wheels on a python pinned to 3.11, none of which the venv
imports. Both containers are optional — absent docker degrades to `None` everywhere.

Registering with a harness takes nothing: the config is committed, one file per harness — see
[the table at the top](#every-harness-no-configuration). Full setup is [INSTALL.md](INSTALL.md).

## Layout

| Path | Contents |
|---|---|
| `run.sh` | one-shot: cmake-build the fork, then launch it windowed with the MCP bridge open |
| `BespokeSynth/` | the synth (submodule) + the C++ bridge in `Source/Mcp*` |
| `mcp/src/bespoke_mcp/` | the Bespoke MCP server (both modes): tools, bridge client, session layer, graph diff, and `analysis/` — `dsp` (the numeric battery), `quality` (LUFS/LRA/true-peak/clipping plus the MIDI-conditioned stem checks and their flags), `semantic` (the Essentia tags), `render` (plots), `pianoroll`, `verdict` |
| `docker/essentia/` + `run-docker-essentia.sh` | the quarantined Essentia container: genre/mood/danceability for one wav, Linux/CPU-only so the same path works from Windows, models bind-mounted read-only from the data pack |
| `docker/audio-lab/` + `run-docker-audio-lab.sh` | the second quarantined container: eight verbs — `triage`, `separate`, `structure`, `transcribe`, the composite `destructure`, `embed`, plus `version`/`selftest`. Verb dispatch, a writable `/out` mount, and progress in `<out>/status.json`; 2.26 GB of torch that never touches the main venv. Delete it and the repo still works |
| `mcp/src/bespoke_mcp/audio_ref/` | the host side of the audio lab: `lab` invokes one verb, parses its result, reads its status file, and returns `None` when docker or the image is absent; `jobs` is the async registry every heavy tool starts work through — `start` returns an id, `status` reads a record from disk so it survives a server restart, progress comes from the container's `status.json`, a cap queues rather than spawning without bound, and **results are paths, never payloads**. Then the reference stack: `manifest` (`references/audio/manifest.json`, the score side's schema and its project-then-library resolution, plus `provenance`), `verify` (the §V battery for audio — magic bytes, HTML-served-as-audio, a full ffmpeg decode, declared-versus-decoded duration, `ok`/`suspect`/`failed`) and `acquire` (the ladder, where **a terminal failure is a result rather than an exception**) |
| `mcp/src/bespoke_mcp/audio_ref/destructure.py` + `exemplars.py` | a recording into a `track_map.json`: material triage first (a mix with no percussive onsets and a harmony that holds skips separation and says so), then stems, structure, transcription and exemplars, assembled host-side so descriptors survive any container stage failing. `exemplars` is the §4.1 algorithm — onsets from the transcription, an isolation gate, scoring, top 3–5 across distinct pitches, −23 LUFS with the gain stored — and `exemplars: []` with a reason is a first-class answer |
| `mcp/src/bespoke_mcp/audio_ref/naming.py` | **whether a `heard_as` top-k is a name worth publishing, and today the answer is always no.** Parses the 444-name vocabulary out of `docs/Sound_Ontology.md` §6, resolves a bank string through `data/bank_label_map.json` (label, then folder deepest-first, then filename group — 90.7 % of the non-pack-01 index reaches one of 118 names), refuses a window measuring more than 1.2 simultaneous pitches as `not_isolated`, and carries the abstain rule. The rule was fitted against a 22-positive / 15-negative control probe and **refused**: 1 correct name in 44, 0 on sitting three's twenty, 1 wrong on the 136 library stems. |
| `mcp/src/bespoke_mcp/audio_ref/index.py` + `soundbank.py` | the **sound bank**, searchable. `index` is the on-disk format (`meta.json`, `items.json`, fp16 blocks already mean-centered, the stored mean beside each, brute-force numpy) and the invariants that make search work rather than merely answer: `embed_query()` is the only way to make a query vector and applies repeat-padding and the stored mean subtraction internally, no public API returns a raw embedding, and a `mean_sha` mismatch raises. `soundbank` is what goes in it — 344 instruments and kit voices read out of `orchestra.json` plus the 14 `.sfz` it cannot see, 4 293 one-shot and loop files, **AKWF's 65 folders as one `granularity: "single-cycle-wavetable"` item each** (hidden from any search that does not name the granularity, because a waveform is evidence about a spectrum and not an instrument), `how_to_play` per result, and `data-packs/sounds/SOUNDS_GUIDE.md` regenerated on every build |
| `tests/fixtures_audio.py` | the audio fixture corpus: a three-source mix synthesised at test time from CC0 pack WAVs, whose stems *are* the inputs — so "did separation recover the bass" is a measurement, not an impression. Plus `committed_fixture()`, the four real files the verification battery needs (a clean mp3, one carrying a real CC-BY attribution frame, a truncated download, an HTML page named `.mp3`) and `encode()`, which transcodes the rest of the container matrix at test time so it costs no committed bytes |
| `score/src/score_mcp/` | the score MCP server: `model` (notes and the meter map), `smf` (mido in/out, history), `engine` (revisions, caps, lints), `regions` (the section DSL), `figures` (the motif verbs), `metrics` (the measurements the mirror and the floors share), `complexity` (the twelve floors), `dynamics` (the hairpin compiler), `views` (roll, overview, text mirror), `notation` (the engraver and the text appendix), `project` (the sidecar both servers share) |
| `tests/python/` | fast unit tests (no synth needed) |
| `tests/integration/` | tests that attach to a running synth, or launch a headless one (content, plugin state, arrangement, observability, the SFZ orchestra) |
| `tests/golden/` | three patches rendered deterministically and asserted note-for-note, plus the determinism test; `retired/` holds three more that were recorded and dropped, with the reason for each |
| `tests/instance.py` | the shared attach-or-launch session fixture, and the blocking `Bridge` JSON-RPC client the integration tests use |
| `BespokeSynth/tests/` | Catch2 unit tests for the NDJSON codec, the event and text rings, and `NamedMutex` |
| `.agents/skills/` | Twenty-seven Claude skills on four axes, with [README.md](.agents/skills/README.md) as the entry point and the reading order. Workflow: patching, sequencing, arranging, modulation, listening. Genre: orchestral, jazz, electronic, rock/metal, hip-hop. Instrument family: drums, bass, keys, strings, brass-winds, guitar. Component/production: synths, sampling, effects, mixing-mastering. Score: songwriting, planning, constraints, writing, dynamics, checking, and **handoff — the flagship for any Bespoke work that starts from a composed score**. Plus `_shared/gotchas.md`, the source-verified trap list every skill points at |
| `data-packs/demos/projects/<name>/` | one directory per piece: `project.json` (plan, tracks, constraints, journal, and the `bespoke` block), `score/` (the `.mid` and its engraving, empty for the 43 legacy demos), `bespoke/` (savestate, audio, analysis, plots) |
| `tools/` | launch, smoke-test, demo-build and golden-recording drivers that speak the bridge directly; `build_listening_battery.py` assembles a sitting of the listening page (sitting one is T2/T4/T5/T6, 170 items; `--sections T6B` is sitting 1b's twenty re-planted controls) |
| `tools/overtone_events.py` | **how an instrument is played**: the event schema, its validator and its Clojure compiler. An event is Bespoke's `NoteMessage` plus its `ModulationParameters` and nothing else — `pitch` as an `int` because Bespoke's is, `bend` in semitones because that is where microtonality rides, `vel`, `voice`, `pressure`, `mod`, `pan` — so one list plays on either side, and the eventual VST is a mapping rather than a translation. `freq`, `dur` and `amp` are always sent; every other field only when the event carries it, which is why declaring a lane on an instrument changes no rendered sample until something sends it |
| `tools/overtone_card.py` | the twenty-one-note card as **data plus its measurement**. `NOTES` is the single source of truth and is compiled when the box is posted, so no `.clj` carries a card and none can drift; `overlap()` is what proves a card is one voice. An instrument needing a performance the shared card cannot express puts a `card.events.json` beside its `.clj` and says in `note` why |
| `tools/overtone_stability.py` | **is a published number the instrument, or one draw of it.** Renders each stem five times, measures every take and puts each one through the whole gate, so a sidecar carries a spread per metric and a pass rate per `@check` rather than a sample. A check whose takes disagree gets a fourth verdict, `unstable`, which is reported and counted and is deliberately **not** a violation — a gate that disagrees with itself is a fact about the test, and the fix is a seed in the `.clj` |
| `tools/overtone_expression.py` | **the card's sibling: one event list that drives what the card never touches.** A trunk every instrument gets — velocity 16 to 127, an octave either side of `@midi`, half a semitone of bend, a short gate against a long one, hard left to hard right — then a tail from that instrument's own `@lane` and `@art` lines. It warns and never gates. Deadness is decided by *subtracting two renders* rather than by a level meter, which is exact because every instrument is reproducible; the slot probe lives here too, and it measures its own floor |
| `overtone/examples/` | the render box's worked examples. `aulos/` is the largest: an ancient Greek double pipe built as **two** `definst`s against two recordings, then used to reconstruct both of them note for note off measured hertz — a quartertone in the second reference means semitones would have lost the mode. [RESEARCH.md](overtone/examples/aulos/RESEARCH.md) carries the physics, the partial tables and how the renders measure back against the recordings; [analyse_refs.py](overtone/examples/aulos/analyse_refs.py) reproduces both note tables row for row |
| `data-packs/` | submodule: 18k sample wavs and 42 curated Linux VST3 bundles, loadable over the bridge |
| `data-packs/sounds/orchestra.json` | the generated SFZ orchestra: 81 instruments × 169 articulation maps + 5 percussion kits, every pitch measured before it was mapped |
| `data-packs/vst-plugins/plugin-profiles.json` | 52 plugins × named parameters + sweet spots, so an agent writes `name="cutoff"` instead of an index |
| `data-packs/vst-plugins/presets/` | 144 auditioned `.vstp` presets indexed by genre role, plus `DELETED.md`, the record of what was pruned |
| `data-packs/models/essentia/` | the vendored genre/mood/danceability model weights and their label files, SHA256-pinned by `tools/fetch_models.py`. **CC BY-NC-SA 4.0 — non-commercial**, with the attribution and source URLs beside them |
| `data-packs/models/audio-lab/` | the audio lab's weights: SHA256-pinned, **permissive only**, and **gitignored** — fetched per host with `--set audio-lab`, because the pack is already at 7.3 GB against its own ceiling. Only the `.gitignore` and the licence README are committed |
| `data-packs/references/` | the shared cross-project reference library: `scores/` (MIDI, committed) and `audio/` (never committed, except an opt-in CC lane with a `LICENSE` beside each item — which is where the four 254 kB verification fixtures live, at `audio/cc/fixtures/`) |
| `AGENTS.md` | the map: what to read first, and which documents to keep current |
| `UPSTREAM.md` | every file touched in the BespokeSynth fork, and why |
| `TODO.md` | roadmap: the module fuzz matrix, CI, bridge features |
| `DEFECTS.md` | the consolidated defect ledger: every open defect from the nine work batches, deduplicated, with a reproduction and a severity — plus what is shipping known-wrong and which claims later measurement retracted |
| `data-packs/README.md` | what the data pack holds and the rules for changing it |
| `docs/Bespoke_Module_Reference.md` | reference for all 234 built-in modules, the Python scripting API, and the bundled example patches |
| `score/README.md` | why `score-mcp` is shaped the way it is: one Standard MIDI File per piece, what lives in the `.mid` versus the sidecar, track 0 as the Conductor Notes track, why mido, the two-registration mode split, and the engraving stack |
| `docs/VST_Fleet_Reference.md` | the measured plugin fleet: parameters, quirks, the state formats behind GUI-only content, what was deleted |

---

## Licence

**AGPL-3.0** ([LICENSE](LICENSE)) — the same licence as [Bespoke Synth](https://github.com/BespokeSynth/BespokeSynth),
which this repository forks and links against.

That covers the code here. The data pack is a submodule with its own terms:
`data-packs/` is AGPL-3.0 for our part of it, while every vendored component — the seven sample
packs, the 42 VST3 plugins, the Essentia models — keeps the licence it arrived with, recorded
next to the files themselves. The Essentia model weights in particular are **CC BY-NC-SA 4.0
(non-commercial)**. See [data-packs/README.md](data-packs/README.md) before redistributing any
of it.