MCP Synth Controller

# MCP Synth Controller This project implements a Python-based **Model Context Protocol (MCP) server** designed to allow large language models (LLMs) to control parameters of a JUCE synthesizer in real time. The MCP server exposes structured *tools* that an LLM can call, and these tool calls are translated into OSC messages sent to a running JUCE plugin or application. This repository contains the **Python side only**. The corresponding JUCE synthesizer must implement an OSC receiver capable of handling messages such as: ``` /setParameter <paramName> <value> ``` The JUCE implementation is being developed separately in: https://github.com/TYLERSFOSTER/MIDIControl001 --- ## Project Overview This project provides: - A working MCP server (`mcp_server.py`) - Tool schemas that define how the LLM may interact (`schemas.py`) - Python functions implementing those tools (`tools.py`) - An OSC bridge for communicating with the JUCE synth (`juce_bridge.py`) - A test suite ensuring correctness (`tests/`) The goal is to enable a workflow such as: 1. A speech-to-text system turns spoken commands into text. 2. The text is fed to an LLM. 3. The LLM responds by invoking MCP tools such as `setParameter`. 4. The MCP server receives the tool call and executes the corresponding Python function. 5. The function sends an OSC message to the JUCE synth, modifying parameters in real time. This architecture allows expressive, natural-language control of a synthesizer using speech or text. --- ## Directory Structure ``` mcp_synth_controller/ ├── pyproject.toml ├── README.md ├── src/ │ └── server/ │ ├── config.py │ ├── juce_bridge.py │ ├── mcp_server.py │ ├── schemas.py │ └── tools.py ├── tests/ │ ├── test_osc_client_init.py │ ├── test_list_parameters.py │ ├── test_set_parameter_message_format.py │ ├── test_mcp_initialize.py │ └── test_mcp_tool_call.py └── examples/ └── test_send_osc.py ``` --- ## Dependency Management This project uses **uv** for Python dependency management. To install dependencies: ``` uv sync ``` To run the MCP server: ``` uv run python src/server/mcp_server.py ``` To run the test suite: ``` uv run pytest ``` --- ## MCP Server The MCP server: 1. Handles the MCP initialization handshake. 2. Advertises available tools to the LLM. 3. Receives tool calls from the LLM. 4. Dispatches these calls to the correct Python functions. 5. Returns structured tool results. The server communicates via STDIN/STDOUT using JSON, following the MCP protocol. --- ## Tools Each tool corresponds to a callable action available to the LLM. Tools currently implemented: ### `setParameter(param: str, value: float)` Sets a synthesizer parameter via OSC. ### `getParameter(param: str)` Placeholder implementation (returns a dummy value). Real bidirectional communication may be added later. ### `listParameters()` Returns a list of known parameters. This can be later expanded to query the JUCE synth dynamically. --- ## OSC Bridge `juce_bridge.py` uses `python-osc` to send messages to JUCE. By default, messages follow this format: ``` /setParameter <paramName> <value> ``` The corresponding JUCE OSCReceiver must be implemented. See the next section. --- ## Required JUCE Implementation To complete this system, the JUCE synth must: 1. Create an `OSCReceiver` 2. Bind to the same port specified in `config.py` 3. Add a listener for `/setParameter` 4. Parse incoming messages and map parameter names to actual JUCE parameters Example responsibilities on the JUCE side: - Initialize an OSCReceiver (`connect(9001)`) - Add a listener for `/setParameter` - Extract parameter name and float value from the OSCMessage - Apply the value using `setValueNotifyingHost` The JUCE implementation belongs in the separate repository: https://github.com/TYLERSFOSTER/MIDIControl001 --- ## Example OSC Test To manually verify OSC transmission: ``` uv run python examples/test_send_osc.py ``` This sends: ``` /setParameter testParam 0.42 ``` If your JUCE OSCReceiver is active, it should appear in your debug output. --- ## Tests This project includes a complete pytest suite validating: - OSC client initialization - OSC message format - MCP initialization handshake - Tool call dispatch logic - Parameter listing behavior Run tests with: ``` uv run pytest ``` All tests should pass. --- ## Future Work - Implement bidirectional OSC or TCP communication with JUCE - Add dynamic parameter discovery from JUCE - Add ramping, smoothing, and modulation utilities - Integrate with speech-to-text pipeline - Provide real-time LLM agent control in Claude Desktop or similar --- ## License MIT License.