<div align="center">
<h1>MCP2OSC</h1>
<h3>Parametric Control by Natural Language</h3>

[Yuan-Yi Fan](https://yuanyifan.com)<sup>1</sup>  

<sup>1</sup> Independent Researcher

</div>

<br>

![mcp2osc_matrix_rotate](https://github.com/user-attachments/assets/29689f81-e540-44cf-b3cb-c1ce35a9b71f)

<br>

[![arXiv](https://img.shields.io/badge/arXiv-2508.10414-b31b1b.svg)](https://arxiv.org/abs/2508.10414)
[![OpenReview](https://img.shields.io/badge/NeurIPS-2025-blue)](https://openreview.net/forum?id=XLdFf7Zarw)
[<img src="https://img.shields.io/badge/YouTube-Video-red.svg?logo=youtube">](https://www.youtube.com/@MCP2OSC)

### Updates
- 2025/11/14: [OpenReview version](https://openreview.net/forum?id=XLdFf7Zarw) is now available. 
- 2025/08/14: [arXiv version](https://arxiv.org/abs/2508.10414) is now public. 

### Abstract 
Text prompts enable intuitive content creation but may fall short in achieving high precision for intricate tasks; knob or slider controls offer precise adjustments at the cost of increased complexity. To address the gap between knobs and prompts, a new MCP (Model Context Protocol) server and a unique set of prompt design criteria are presented to enable exploring parametric OSC (OpenSoundControl) control by natural language prompts. Demonstrated by 15 practical QA examples with best practices and the generalized prompt templates, this study finds Claude integrated with the MCP2OSC server effective in generating OSC messages by natural language, interpreting, searching, and visualizing OSC messages, validating and debugging OSC messages, and managing OSC address patterns. MCP2OSC enhances human-machine collaboration by harnessing a LLM (Large Language Model) to handle intricate OSC development tasks. It empowers human creativity with an intuitive language interface featuring flexible precision controls: a promptbased OSC tool. This study provides a novel perspective on the creative MCP application at the network protocol level by utilizing LLM’s strength in directly processing and generating human-readable OSC messages. The results suggest its potential for a LLM-based universal control mechanism for multimedia devices.

### Demo video
[![Watch the video](https://img.youtube.com/vi/LquHSm8Lquo/sddefault.jpg)](https://www.youtube.com/watch?v=LquHSm8Lquo)
[![Watch the video](https://img.youtube.com/vi/qQ23gsT5TvE/sddefault.jpg)](https://www.youtube.com/watch?v=qQ23gsT5TvE)
[![Watch the video](https://img.youtube.com/vi/O0VdbRiggfg/sddefault.jpg)](https://www.youtube.com/watch?v=O0VdbRiggfg)
[![Watch the video](https://img.youtube.com/vi/t1qOGX-SDuU/sddefault.jpg)](https://www.youtube.com/watch?v=t1qOGX-SDuU)

### Installation

1. **Clone the repository**
   ```bash
   git clone https://github.com/yyf/MCP2OSC.git
   cd MCP2OSC
   ```

2. **Install dependencies**
   ```bash
   npm install
   ```

3. **Configure your LLM application**
   Add to your Claude Desktop MCP configuration file — for example:
   - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`   

   Use the **absolute path** to `mcp-server.js` on your machine.
   ```json
   {
      "mcpServers": {
         "mcp2osc": {
            "command": "node",
            "args": ["/Users/.../MCP2OSC/mcp-server.js"],
            "cwd": "/Users/.../MCP2OSC",
            "env": {
            "OSC_HOST": "127.0.0.1",
            "OSC_SEND_PORT": "9500",
            "OSC_RECEIVE_PORT": "9501", 
            "WEBSOCKET_PORT": "8765",
            "WEBSOCKET_OSC_PORT": "9500",
            "MAX_OSC_MESSAGES": "2000",
            "OSC_LOG_ROTATION": "true"
            }
         }
      }
   }
   ```

4. **Start the LLM application, i.e. Claude desktop application**   
- Claude desktop application can be downloaded here: https://claude.ai/download
- Restart Claude if you make any change to the claude config json file 
- Open the example MaxMSP or PureData patch and try the example prompts in Claude

5. **Example prompts**   
- "What MCP tools are available"
- "Ping MaxMSP at port 9500 then check if there is any response"
- "Check received OSC messages"

6. **Optional: start MCP + dashboard**
   ```bash
   npm start
   ```
7. **Creative application examples**
- The examples folder include MaxMSP, PureData, and SuperCollider patches

### Citation
```
@inproceedings{
fan2025mcposc,
title={{MCP}2{OSC}: Parametric Control by Natural Language},
author={Yuan-Yi Fan},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Creative AI Track: Humanity},
year={2025},
url={https://openreview.net/forum?id=XLdFf7Zarw}
}
```

### ❤️ Made with MCP2OSC 🎛️ - a selection of projects exploring the possibilities of MCP2OSC
| Project | Author | Description |
|---|---|---|
| [coralline](https://github.com/olivelogs/coralline) | [olivelogs](https://github.com/olivelogs) | Coralline gives Claude the ability to make music in real-time with SuperCollider, effectively translating dialogue into music. |