# Reachy Mini MCP Server

[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=agentculture_reachy-mini-mcp&metric=coverage)](https://sonarcloud.io/summary/new_code?id=agentculture_reachy-mini-mcp)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=agentculture_reachy-mini-mcp&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=agentculture_reachy-mini-mcp)

<!-- Badges activate once the SonarCloud project `agentculture_reachy-mini-mcp` and
the `SONAR_TOKEN` repo secret are provisioned; see CLAUDE.md. -->

A Model Context Protocol (MCP) server for controlling the [Reachy Mini](https://github.com/pollen-robotics/reachy_mini) robot using [FastMCP](https://github.com/jlowin/fastmcp).

> [!NOTE]
> **Looking for the full Conversation App?**
> This repository also contains the full "Conversation Stack" (Hearing + LLM + Conversation Logic) which turns Reachy Mini into an autonomous conversational robot.
>
> The **Docker setup** is specifically for running this full conversation application.
>
> 👉 **[Read the Conversation Stack Overview](docs/conversation_stack.md)**
>
> 👉 **[Full Docker Setup Guide](DOCKER_SETUP.md)**

This MCP server provides a comprehensive set of tools to control Reachy Mini's head movements, antennas, camera, and perform various gestures and emotional expressions.

## Roadmap

### MCP

- [ ] Speech: TTS, piper
- [ ] Agentic layer
- [ ] Queue actions support
- [ ] Interruption support
- [ ] Move body support, reachy-mini python SDK

### Conversation app

- [ ] Hearing:
  - [ ] VAD, WebRTCVAD / SileroVAD
  - [ ] STT, faster-whisper
- [ ] Vision
- [ ] Memory:
  - [ ] File
  - [ ] Vector DB
  - [ ] Vector Graph DB
- [ ] Growth, nightly fine tunes support
- [ ] MQTT support
- [ ] Move to [autonomous-intelligence](https://github.com/OriNachum/autonomous-intelligence) repo

## Features

### Movement & Speech Control

- **Head Control**: Move the head in 3D space (x, y, z) with orientation (roll, pitch, yaw)
- **Antenna Control**: Control left and right antennas independently
- **Speech (TTS)**: Make the robot speak while performing actions (via `speech` parameter)
- **Gestures**: Perform predefined gestures (greeting, yes, no, thinking, celebration)
- **Emotions**: Express emotions (happy, sad, curious, surprised, confused)
- **Direction Looking**: Make the robot look in specific directions (up, down, left, right)

### Monitoring & Control

- **State Monitoring**: Get full robot state, head state, antenna state, camera state
- **Power Management**: Turn robot on/off
- **Emergency Stop**: Immediately halt all movements
- **Health Status**: Monitor robot health and system status

### Advanced Features

- **Command Sequences**: Execute multiple robot operations in a single call (NEW!)
- **Single Unified Interface**: Access all functionality through one `operate_robot` tool
- **Repository-Based Tools**: Easily extensible tool system with JSON definitions

## Prerequisites

1. **Python 3.10+** (tested with Python 3.10-3.13)
2. **Reachy Mini Robot**: Either physical robot connected via USB or wireless, or simulation running in MuJoCo
3. **Reachy Mini Daemon**: Must be running on `localhost:8000` (default)

## Installation

The project ships as a pip-installable distribution, `reachy-mini-mcp`, which
provides both the MCP **server** and a manager **CLI** (`reachy-mini-mcp`).

```bash
# Manager CLI only (pure stdlib — show/install/uninstall/doctor/overview):
uv tool install reachy-mini-mcp        # or: pipx install reachy-mini-mcp

# To actually run the server, add the [server] extra (and [tts] for speech):
pip install "reachy-mini-mcp[server]"          # MCP stdio server
pip install "reachy-mini-mcp[server,tts]"      # + Piper TTS
pip install "reachy-mini-mcp[openai]"          # OpenAI-compatible HTTP server
```

From a source checkout, `./setup.sh` creates a `.venv` and installs the package
editable with `[server,tts]`.

## The manager CLI

`reachy-mini-mcp` is an MCP-server manager — it tells an agent or operator how to
register and run the server, and can do it for them:

```bash
reachy-mini-mcp overview     # one-screen status (also the no-arg default)
reachy-mini-mcp show         # print the mcp.json snippet for this machine
reachy-mini-mcp explain      # how MCP registration works + where configs live
reachy-mini-mcp install --client claude-code --scope project   # set it up
reachy-mini-mcp uninstall --client claude-code --scope project # put it down
reachy-mini-mcp doctor       # diagnose deps, daemon, and client registration
reachy-mini-mcp serve        # run the FastMCP stdio server (what mcp.json calls)
```

`install` / `uninstall` target `claude-code` (project `.mcp.json` or user
`~/.claude.json`), `claude-desktop`, `cursor`, or any `--path FILE`. They merge
into existing config (other servers are preserved) and accept `--dry-run` to
print the result without writing.

## Registering with an MCP client

The `mcp.json` entry to add (also produced by `reachy-mini-mcp show`):

```json
{
  "mcpServers": {
    "reachy-mini": {
      "command": "reachy-mini-mcp",
      "args": ["serve"],
      "env": { "REACHY_BASE_URL": "http://localhost:8000" }
    }
  }
}
```

## Running the MCP Server

### Step 1: Start the Reachy Mini Daemon

Before starting the MCP server, you need to have the Reachy Mini daemon running.

👉 **[Follow the official Reachy Mini Daemon Setup Guide](https://github.com/pollen-robotics/reachy_mini)**

Ensure the daemon is running and accessible (default: `http://localhost:8000`).

### Step 2: Start the MCP Server

```bash
reachy-mini-mcp serve
```

Equivalent invocations: `python -m reachy_mini_mcp serve`, or for a source
checkout `fastmcp run reachy_mini_mcp/server.py`.

The MCP server will now be running and ready to accept connections from MCP clients.

## Available MCP Tools

### Single Unified Tool: `operate_robot`

This MCP server exposes **one MCP tool** that provides access to all robot control functionality:

| Tool | Description |
|------|-------------|
| `operate_robot(tool_name, parameters)` | **Meta-tool** to dynamically execute any robot control operation by name. |
| `operate_robot(commands)` | **Sequence mode** to execute multiple operations in sequence. |

This unified interface allows you to call any of the robot control operations either individually or as a sequence:

**Single Command Mode:**

```python
# Example: Get robot state
operate_robot("get_robot_state")

# Example: Express emotion with parameters
operate_robot("express_emotion", {"emotion": "happy"})

# Example: Move head with multiple parameters
operate_robot("move_head", {"z": 10, "duration": 2.0, "mm": True})
```

**Sequence Mode (NEW!):**

```python
# Example: Execute multiple commands in sequence
operate_robot(commands=[
    {"tool_name": "perform_gesture", "parameters": {"gesture": "greeting"}},
    {"tool_name": "nod_head", "parameters": {"duration": 2.0, "angle": 15}},
    {"tool_name": "move_antennas", "parameters": {"left": 30, "right": -30, "duration": 1.5}},
    {"tool_name": "look_at_direction", "parameters": {"direction": "left", "duration": 1.0}}
])
```

**Note:** The tool name must match exactly. The correct tool is `get_robot_state`, not `get_robot_status`.

### Available Robot Operations

All operations are accessible through the `operate_robot` tool. Here are all available operations:

#### Basic State & Control

| Operation | Description |
|-----------|-------------|
| `get_robot_state` | Get full robot state including all components |
| `get_head_state` | Get current head position and orientation |
| `get_antennas_state` | Get current antenna positions |
| `get_camera_state` | Get camera status |
| `get_power_state` | Check if robot is powered on/off |
| `get_health_status` | Get overall health status |
| `turn_on_robot` | Power on the robot |
| `turn_off_robot` | Power off the robot |
| `stop_all_movements` | Emergency stop all movements |

#### Head Movement

| Operation | Description |
|-----------|-------------|
| `move_head` | Move head to specific pose (params: x, y, z, roll, pitch, yaw, duration) |
| `reset_head` | Return head to neutral position |
| `nod_head` | Make robot nod (params: duration, angle) |
| `shake_head` | Make robot shake head (params: duration, angle) |
| `tilt_head` | Tilt head left or right (params: direction, angle, duration) |
| `look_at_direction` | Look in a direction (params: direction - up/down/left/right, duration) |

#### Antenna Movement

| Operation | Description |
|-----------|-------------|
| `move_antennas` | Move antennas to specific positions (params: left, right, duration) |
| `reset_antennas` | Return antennas to neutral position |

#### Emotions & Gestures

| Operation | Description |
|-----------|-------------|
| `express_emotion` | Express emotion (params: emotion - happy/sad/curious/surprised/confused) |
| `perform_gesture` | Perform gesture (params: gesture - greeting/yes/no/thinking/celebration) |

#### Camera

| Operation | Description |
|-----------|-------------|
| `get_camera_image` | Capture image from camera |
| `get_camera_state` | Get camera status |

## Usage Examples

All operations are called through the `operate_robot` tool. Here are some examples:

### Example 1: Basic Head Movement

```python
# In your MCP client (e.g., Claude Desktop)
# Move head up 10mm and tilt 15 degrees
operate_robot("move_head", {"z": 10, "roll": 15, "duration": 2.0})

# Return to neutral
operate_robot("reset_head")
```

### Example 2: Express Emotions

```python
# Make the robot look happy
operate_robot("express_emotion", {"emotion": "happy"})

# Make the robot look curious
operate_robot("express_emotion", {"emotion": "curious"})

# Return to neutral
operate_robot("express_emotion", {"emotion": "neutral"})
```

### Example 3: Perform Gestures

```python
# Wave hello
operate_robot("perform_gesture", {"gesture": "greeting"})

# Nod yes
operate_robot("perform_gesture", {"gesture": "yes"})

# Shake no
operate_robot("perform_gesture", {"gesture": "no"})
```

### Example 4: Complex Interaction

```python
# Turn on the robot
operate_robot("turn_on_robot")

# Check state
state = operate_robot("get_robot_state")

# Make robot look around
operate_robot("look_at_direction", {"direction": "left", "duration": 1.5})
operate_robot("look_at_direction", {"direction": "right", "duration": 1.5})
operate_robot("look_at_direction", {"direction": "forward"})

# Express surprise
operate_robot("express_emotion", {"emotion": "surprised"})

# Perform celebration
operate_robot("perform_gesture", {"gesture": "celebration"})

# Turn off when done
operate_robot("turn_off_robot")
```

### Example 5: Control Antennas

```python
# Move antennas independently
operate_robot("move_antennas", {"left": 30, "right": -30, "duration": 1.0})

# Reset to neutral
operate_robot("reset_antennas")
```

### Example 6: Command Sequences (NEW!)

Execute complex robot behaviors with command sequences:

```python
# Greeting sequence
operate_robot(commands=[
    {"tool_name": "express_emotion", "parameters": {"emotion": "happy"}},
    {"tool_name": "perform_gesture", "parameters": {"gesture": "greeting"}},
    {"tool_name": "nod_head", "parameters": {"duration": 1.5, "angle": 10}},
    {"tool_name": "reset_head", "parameters": {}}
])

# Curious behavior - look around
operate_robot(commands=[
    {"tool_name": "express_emotion", "parameters": {"emotion": "curious"}},
    {"tool_name": "move_antennas", "parameters": {"left": 30, "right": 30, "duration": 1.0}},
    {"tool_name": "look_at_direction", "parameters": {"direction": "left", "duration": 1.0}},
    {"tool_name": "look_at_direction", "parameters": {"direction": "right", "duration": 1.0}},
    {"tool_name": "look_at_direction", "parameters": {"direction": "forward", "duration": 0.5}}
])

# Initialization routine
operate_robot(commands=[
    {"tool_name": "turn_on_robot", "parameters": {}},
    {"tool_name": "reset_head", "parameters": {}},
    {"tool_name": "reset_antennas", "parameters": {}},
    {"tool_name": "get_robot_state", "parameters": {}}
])
```

For more details on command sequences, see [SEQUENCE_COMMANDS.md](SEQUENCE_COMMANDS.md).

## Using with MCP Supported client

The easiest way is to let the manager CLI write the config for you:

```bash
reachy-mini-mcp install --client claude-desktop   # or claude-code / cursor
```

To do it by hand, add this to your client's config file (the same JSON works for
Claude Desktop, Claude Code, and Cursor — see locations below):

```json
{
  "mcpServers": {
    "reachy-mini": {
      "command": "reachy-mini-mcp",
      "args": ["serve"],
      "env": { "REACHY_BASE_URL": "http://localhost:8000" }
    }
  }
}
```

Config file locations:

- **Claude Desktop** — macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`;
  Linux: `~/.config/Claude/claude_desktop_config.json`;
  Windows: `%APPDATA%\Claude\claude_desktop_config.json`
- **Claude Code** — project: `./.mcp.json`; user: `~/.claude.json`
- **Cursor** — project: `./.cursor/mcp.json`; user: `~/.cursor/mcp.json`

After editing the config, restart the client. The Reachy Mini tools will be available in your conversations.

## MCP Prompts

The server includes helpful prompts:

- `control_prompt` - Guidelines for controlling Reachy Mini
- `safety_prompt` - Safety guidelines and limits

## Architecture

```text
┌─────────────────┐         ┌──────────────────┐         ┌─────────────────┐
│  MCP Client     │◄───────►│  FastMCP Server  │◄───────►│ Reachy Daemon   │
│  (Claude, etc)  │  stdio  │ (reachy-mini-mcp │  HTTP   │  (localhost:8000)│
│                 │         │      serve)      │         │                 │
└─────────────────┘         └──────────────────┘         └─────────────────┘
                                                                    │
                                                                    ▼
                                                          ┌─────────────────┐
                                                          │  Reachy Mini    │
                                                          │  Robot/Sim      │
                                                          └─────────────────┘
```

## Development

### Repository-Based Tool System

This MCP server uses a **repository-based approach** for defining tools, making it highly extensible and customizable. Tools are defined in JSON files rather than hardcoded in Python.

**Key Benefits:**

- ✅ Add new tools without modifying server code
- ✅ Customize tool behavior by editing JSON files
- ✅ Easy to version control and share tool definitions
- ✅ Script-based execution for complex operations

**Repository Structure:**

```text
tools_repository/
├── tools_index.json          # Root file listing all tools
├── *.json                    # Individual tool definitions
└── scripts/                  # Python scripts for complex tools
    ├── nod_head.py
    ├── shake_head.py
    ├── express_emotion.py
    └── perform_gesture.py
```

### Adding New Tools

Create a Python script in `tools_repository/scripts/my_tool.py`:

```python
async def execute(make_request, create_head_pose, params):
    """Execute the tool."""
    # Your logic here
    await make_request("POST", "/api/endpoint1", json_data={...})
    await asyncio.sleep(1.0)
    await make_request("POST", "/api/endpoint2", json_data={...})
    return {"status": "success"}
```

Then create a JSON file (e.g., `tools_repository/my_tool.json`):

```json
{
  "name": "my_tool",
  "description": "Description of what my tool does",
  "parameters": {
    "required": [
      {"name": "param1", "type": "string", "description": "First parameter"}
    ],
    "optional": [
      {"name": "param2", "type": "number", "default": 1.0, "description": "Second parameter"}
    ]
  },
  "execution": {
    "type": "script",
    "script_file": "my_tool.py"
  }
}
```

Add to `tools_repository/tools_index.json`:

```json
{
  "name": "my_tool",
  "enabled": true,
  "definition_file": "my_tool.json"
}
```

Restart the server - your tool is now available!

### Testing Tools

Validate your tool definitions:

```bash
python test_repository.py
```

This verifies all JSON files are valid and script files exist.

## Resources

- [Reachy Mini GitHub](https://github.com/pollen-robotics/reachy_mini)
- [FastMCP Documentation](https://github.com/jlowin/fastmcp)
- [Model Context Protocol Specification](https://modelcontextprotocol.io/)

## License

This project is licensed under the MIT License, including the conversation and hearing app within this repository, and does not extend to the Reachy Mini Daemon.

## Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

## Support

For issues related to:

- **This MCP server**: Open an issue in this repository
- **Reachy Mini robot**: Visit [Reachy Mini GitHub Issues](https://github.com/pollen-robotics/reachy_mini/issues)

## Credits

- Built with [FastMCP](https://github.com/jlowin/fastmcp) by Marvin
- For [Reachy Mini](https://github.com/pollen-robotics/reachy_mini) by Pollen Robotics & Hugging Face
- Follows the [Model Context Protocol](https://modelcontextprotocol.io/) specification