# MCP Server Setup Guide
This guide explains how to set up the MCP servers required for **real** robotics webapp functionality (not mock data).
## ⚠️ Important: Mock Data vs Real Data
**By default, the robotics webapp uses MOCK DATA** for demonstration purposes. To enable real integration with Unity3D, VRChat, Avatar tools, OSC, VRoid, Resonite, and AI/LLM features, you must:
1. Install and configure each MCP server
2. Start the MCP servers on their configured ports
3. Ensure the backend can connect to them
**The webapp will display clear warnings when mock data is active.**
## Prerequisites
### Hardware Requirements
- **Multiple Monitors**: **2+ monitors strongly recommended** for optimal work environment
- **Minimum**: 2 monitors (webapp + one environment app)
- **Recommended**: 3-4 monitors (webapp + Unity3D + Resonite + VRChat/VRoid)
- **Optimal**: 4+ monitors for full multi-app workflow
- **Why**: Enables simultaneous visibility of webapp controls and all environment applications without alt-tabbing
- **Windows 10/11** with virtual desktop support (for multi-desktop features)
### Software Requirements
- Python 3.8+ installed
- Node.js and npm (for frontend)
- Access to the respective applications (Unity, VRChat, etc.)
## MCP Server Ports
| Server | Default Port | Environment Variable | Repository |
|--------|--------------|---------------------|------------|
| Unity3D | 8001 | `UNITY3D_MCP_URL` | `unity3d-mcp` |
| VRChat | 8002 | `VRCHAT_MCP_URL` | `vrchat-mcp` |
| Avatar | 8003 | `AVATAR_MCP_URL` | `avatar-mcp` |
| OSC | 8004 | `OSC_MCP_URL` | `osc-mcp` |
| VRoid | 8005 | `VROID_MCP_URL` | `vroidstudio-mcp` |
| Resonite | 8006 | `RESONITE_MCP_URL` | `resonite-mcp` |
| Local LLM | 8007 | `LOCAL_LLM_MCP_URL` | `local-llm-mcp` |
## Quick Setup (All Servers)
### 1. Clone MCP Server Repositories
```powershell
cd D:\Dev\repos
git clone https://github.com/sandraschi/unity3d-mcp.git
git clone https://github.com/sandraschi/vrchat-mcp.git
git clone https://github.com/sandraschi/avatar-mcp.git
git clone https://github.com/sandraschi/osc-mcp.git
git clone https://github.com/sandraschi/vroidstudio-mcp.git
git clone https://github.com/sandraschi/resonite-mcp.git
```
### 2. Install Dependencies
For each MCP server:
```powershell
cd unity3d-mcp
pip install -e ".[dev]"
cd ..\vrchat-mcp
pip install -e ".[dev]"
cd ..\avatar-mcp
pip install -e ".[dev]"
cd ..\osc-mcp
pip install -e ".[dev]"
cd ..\vroidstudio-mcp
pip install -e ".[dev]"
cd ..\resonite-mcp
pip install -e ".[dev]"
cd ..\local-llm-mcp
pip install -e ".[dev]"
```
### 3. Start MCP Servers in HTTP Mode
Each MCP server needs to run in HTTP mode (not stdio) for webapp integration:
#### Unity3D MCP Server
```powershell
cd D:\Dev\repos\unity3d-mcp
python -m unity3d_mcp.server --host 127.0.0.1 --port 8001
```
#### VRChat MCP Server
```powershell
cd D:\Dev\repos\vrchat-mcp
vrchat-mcp --host 127.0.0.1 --port 8002
```
#### Avatar MCP Server
```powershell
cd D:\Dev\repos\avatar-mcp
python -m avatar_mcp.server --host 127.0.0.1 --port 8003
```
#### OSC MCP Server
```powershell
cd D:\Dev\repos\osc-mcp
python -m oscmcp.server --host 127.0.0.1 --port 8004
```
#### VRoid Studio MCP Server
```powershell
cd D:\Dev\repos\vroidstudio-mcp
python -m vroidstudio_mcp.server --host 127.0.0.1 --port 8005
```
#### Resonite MCP Server
```powershell
cd D:\Dev\repos\resonite-mcp
resonite-mcp --host 127.0.0.1 --port 8006
```
### 4. Verify Server Health
Check that each server is running:
```powershell
# Unity3D
curl http://localhost:8001/health
# VRChat
curl http://localhost:8002/health
# Avatar
curl http://localhost:8003/health
# OSC
curl http://localhost:8004/health
# VRoid
curl http://localhost:8005/health
# Resonite
curl http://localhost:8006/health
# Local LLM
curl http://localhost:8007/health
```
### 5. Configure Backend Environment
Create `.env` file in `robotics-webapp/backend/`:
```env
UNITY3D_MCP_URL=http://localhost:8001
VRCHAT_MCP_URL=http://localhost:8002
AVATAR_MCP_URL=http://localhost:8003
OSC_MCP_URL=http://localhost:8004
VROID_MCP_URL=http://localhost:8005
RESONITE_MCP_URL=http://localhost:8006
LOCAL_LLM_MCP_URL=http://localhost:8007
```
### 6. Start Robotics Webapp Backend
```powershell
cd D:\Dev\repos\robotics-webapp\backend
python main.py
```
The backend will automatically check MCP server health on startup.
### 7. Start Robotics Webapp Frontend
```powershell
cd D:\Dev\repos\robotics-webapp
npm run dev
```
## Individual Server Setup
### Unity3D MCP Server
**Requirements:**
- Unity Editor installed
- Unity project path configured
**Configuration:**
```env
UNITY_PROJECT_PATH=C:/UnityProjects/RoboticsSimulation
UNITY_EDITOR_PATH=C:/Program Files/Unity/Hub/Editor/2022.3.15f1/Editor/Unity.exe
```
**Features:**
- Scene management
- Build automation
- Component manipulation
- Physics configuration
### VRChat MCP Server
**Requirements:**
- VRChat client installed
- OSC enabled in VRChat settings (default port 9000)
**Configuration:**
```env
VRCHAT_OSC_PORT=9000
VRCHAT_API_KEY=your_api_key_here
```
**Features:**
- Avatar control
- Parameter management
- World management
- Social features
### Avatar MCP Server
**Requirements:**
- VRM 2.0 models
- Python 3.9+
**Configuration:**
```env
AVATAR_MODELS_PATH=C:/Avatars
AVATAR_CACHE_PATH=C:/Avatars/cache
```
**Features:**
- VRM loading and management
- Animation control
- Bone manipulation
- Blend shape control
### OSC MCP Server
**Requirements:**
- OSC-enabled applications (Ableton Live, TouchDesigner, etc.)
**Configuration:**
```env
OSC_DEFAULT_PORT=9001
OSC_SEND_PORT=9000
```
**Features:**
- Bidirectional OSC communication
- Application control
- Parameter monitoring
### VRoid Studio MCP Server
**Requirements:**
- VRoid Studio installed
- Windows COM automation support
**Configuration:**
```env
VROIDSTUDIO_PATH=C:\Program Files\VRoidStudio\VRoidStudio.exe
```
**Features:**
- Character creation automation
- Export management
- Rigging control
### Resonite MCP Server
**Requirements:**
- Resonite client installed
- OSC enabled in Resonite (default port 9000)
**Configuration:**
```env
RESONITE_OSC_PORT=9000
RESONITE_USER_ID=your_user_id
```
**Features:**
- World management
- Avatar control
- ProtoFlux scripting
- Social integration
## Verification Checklist
- [ ] All MCP servers installed
- [ ] All MCP servers running on correct ports (8001-8007)
- [ ] Health checks passing (`/health` endpoints)
- [ ] Backend `.env` configured with all MCP URLs
- [ ] Backend started and connecting to MCP servers
- [ ] Frontend showing "Connected" status (not "Mock Data Mode")
- [ ] Real tool calls working (not falling back to mocks)
- [ ] Local LLM MCP server running (for AI features)
- [ ] Ollama/LM Studio running (if using local models)
## Troubleshooting
### Server Not Connecting
1. **Check if server is running:**
```powershell
netstat -an | findstr "8001 8002 8003 8004 8005 8006"
```
2. **Check server logs** for errors
3. **Verify port conflicts** - ensure no other applications using these ports
4. **Check firewall** - ensure ports are not blocked
### Mock Data Still Showing
1. **Check backend logs** for connection errors
2. **Verify environment variables** are set correctly
3. **Restart backend** after changing `.env`
4. **Check browser console** for API errors
### Tool Calls Failing
1. **Check MCP server logs** for tool execution errors
2. **Verify tool names** match MCP server API
3. **Check tool parameters** are correct format
4. **Review MCP server documentation** for tool requirements
## Running All Servers at Once
Create a PowerShell script `start-all-mcp-servers.ps1`:
```powershell
# Start all MCP servers in separate windows
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\unity3d-mcp; python -m unity3d_mcp.server --host 127.0.0.1 --port 8001"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\vrchat-mcp; vrchat-mcp --host 127.0.0.1 --port 8002"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\avatar-mcp; python -m avatar_mcp.server --host 127.0.0.1 --port 8003"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\osc-mcp; python -m oscmcp.server --host 127.0.0.1 --port 8004"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\vroidstudio-mcp; python -m vroidstudio_mcp.server --host 127.0.0.1 --port 8005"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\resonite-mcp; resonite-mcp --host 127.0.0.1 --port 8006"
Start-Process powershell -ArgumentList "-NoExit", "-Command", "cd D:\Dev\repos\local-llm-mcp; python -m src.llm_mcp.main --host 127.0.0.1 --port 8007"
Write-Host "All MCP servers starting in separate windows..."
Write-Host "Wait for all servers to show 'Application startup complete' before using the webapp"
```
## Status Indicators in Webapp
The webapp displays clear status indicators:
- **🟢 Green Banner**: "MCP Server Connected - Using real data"
- **🔴 Red Banner**: "MOCK DATA MODE - MCP Server Not Connected"
- **🟡 Yellow Banner**: "Checking connection..."
Each page shows its specific MCP server status at the top.
## Next Steps
Once all servers are running:
1. Open the robotics webapp
2. Check each page for connection status
3. Verify real data is loading (not mock data)
4. Test tool calls to ensure they work
For detailed API documentation, see each MCP server's README.md file.
