# Unity MCP Server

A Model Context Protocol (MCP) server that lets an AI assistant drive the Unity Editor: scenes, GameObjects,
assets, builds, profiling, play-mode testing and more. It is a Node/TypeScript server that talks over a local
WebSocket to a C# plugin running inside the Editor. It also carries tooling for XREAL glasses development, which
is being reworked for XREAL SDK 3.x (see [XREAL support](#xreal-support-in-progress)).

**82 tools, 17 resources, 13 prompts.** Not every tool is complete: 44 tools are real, 31 are partial,
and 7 XREAL tools return an explicit "not implemented" or "not available" error. Each tool's status is listed
below and, with notes, in [`.claude/CLAUDE.md`](.claude/CLAUDE.md). Where the code and this README disagree,
the code is right; please open an issue.

## Status

Statuses come from [`docs/HANDLER_AUDIT.md`](docs/HANDLER_AUDIT.md), a read-through of every C# handler, updated for
the fixes made afterwards. They describe what the code does; most of it has not been re-tested in an Editor
since, so "real" does not mean bug-free.

| Status | Meaning |
|---|---|
| Real | Performs the Unity operation and reports truthfully. |
| Partial | Works, but ignores some parameters, is narrower than its schema, or some actions are missing. |
| Not available | Returns an error: the data exists only on a device or a running XR subsystem, not in the Editor. |
| Not implemented | Returns an error and changes nothing. |

Since the 2026-10-10 honesty pass no tool returns simulated or invented data; a tool that cannot do something says
so in its error text. Remaining cases where a setter skips a value but still reports success are called out in the
Partial notes.

## Security model

The bridge can run arbitrary C# in the Editor (`execute_code`) and read and write project files
(`file_operations`), so it is local-only by design.

- The Unity plugin listens on **127.0.0.1 only** (default port 8090). It is not reachable from the network.
  Setting `UNITY_HOST` to another machine will not connect.
- WebSocket handshakes that carry a **browser `Origin`** (a URL, or `null`) are rejected, so a web page cannot
  reach the bridge, including through DNS rebinding. Clients that send no `Origin`, such as the Node server, connect.
- There is **no token or per-client approval yet**: any process on the same machine can connect. Do not forward
  or tunnel the port to untrusted hosts, and do not run it on a shared machine you do not trust.
- The Editor plugin starts automatically in the interactive Editor and stays off in `-batchmode` unless started with
  `-mcpServer` (see [Unity command-line flags](#unity-command-line-flags)).

## Tools

Grouped by area and status. The XR tools were run in an Editor on 2026-10-11 but not yet on a device. Per-tool notes are in
[`.claude/CLAUDE.md`](.claude/CLAUDE.md).

**Scenes and GameObjects (11)**

- Real: `create_scene`, `delete_scene`, `load_scene`, `get_gameobject`, `delete_gameobject`, `duplicate_gameobject`, `find_gameobjects`, `select_gameobject`, `editor_selection`, `update_component`
- Partial: `update_gameobject`

**Assets, packages and Hub (11)**

- Real: `manage_asset`, `add_package`, `add_external_dll`, `asset_store`, `file_operations`, `unity_hub`
- Partial: `add_asset_to_scene`, `create_prefab`, `prefab`, `scriptable_object`, `asset_import`

**Editor, play mode and testing (13)**

- Real: `execute_menu_item`, `play_mode`, `send_console_log`, `get_console_logs`, `execute_code`, `run_tests`, `play_sequence`, `playtest`, `restart_server`
- Partial: `recompile_scripts`, `watch_console`, `editor_control`, `undo_redo`

**Build, project and diagnostics (5)**

- Partial: `build_pipeline`, `project_settings`, `script_management`, `profiler`, `debugger`

**Domain systems (11)**

- Real: `audio_mixer`, `physics2d`, `terrain`, `tilemap`
- Partial: `animation`, `lighting`, `material_shader`, `navmesh`, `particle_system`, `physics`, `sprite`

**Knowledge (1)**

- Real: `search_unity_knowledge`

**XREAL: project setup (5)**

- Real: `install_xreal_sdk`, `setup_xreal_project`, `validate_xreal_setup`, `configure_xreal_settings`
- Partial: `configure_android_build`

**XREAL: device (3)**

- Real: `set_tracking_mode` (edit-time initial tracking type)
- Partial: `get_camera_frame`
- Not available: `get_xreal_device_info`

**XREAL: hand tracking (3)**

- Real: `enable_hand_tracking` (refused for One / One Pro unless forced), `create_hand_interactable`
- Not available: `get_hand_state`

**XREAL: spatial mapping (5)**

- Real: `enable_plane_detection`
- Partial: `create_spatial_anchor`, `manage_spatial_anchors`
- Not available: `get_detected_planes`
- Not implemented: `enable_meshing`

**XREAL: image tracking (3)**

- Not available: `get_tracked_images`
- Not implemented: `add_tracking_image`, `configure_image_tracking`

**XREAL: display (1)**

- Real: `configure_xreal_display`

**XREAL: build and deploy (3)**

- Real: `get_build_status`
- Partial: `build_xreal_apk`, `get_connected_devices`

**XREAL: XR interaction (4)**

- Real: `setup_xr_interaction`, `create_xr_rig`, `add_xr_interactor`, `create_xr_ui`

**XREAL: performance (3)**

- Partial: `get_xr_performance_metrics`, `profile_xr_scene`, `capture_xr_screenshot`

## Resources (17)

- Real: `unity://scenes_hierarchy`, `unity://gameobject/{idOrName}`, `unity://logs/{logType}`, `unity://packages`,
  `unity://editor_state`, `xreal://spatial_anchors`, `xreal://build_settings`
- Partial: `unity://menu-items` (hard-coded fallback list), `unity://assets` (four folders, 50 entries each),
  `unity://tests/{testMode}`, `unity://profiler/{dataType}`, `unity://build/{infoType}`, `unity://settings/{category}`
- Not available: `xreal://device_state`, `xreal://hand_tracking/{hand}`, `xreal://detected_planes`,
  `xreal://tracked_images`

## Prompts (13)

- Workflow guidance: `gameobject_handling_strategy`, `script_workflow`, `scene_setup`, `debugging_workflow`,
  `performance_optimization`, `ui_development`, `2d_game_development`, `world_building`, `audio_design`
- XREAL (rewritten for SDK 3.x, 2026-10-11): `xreal_project_setup`, `hand_interaction_strategy`, `spatial_anchor_workflow`,
  `xreal_optimization_guide`. They use only tools that exist and state what the One Pro lacks.

---

## XREAL support (in progress)

The XREAL tools were first written against the legacy NRSDK, and many of them returned simulated or echoed
results. Since 2026-10-10 each one either does real Editor-side work, reports what it skipped, or fails with an
explicit error.

**Direction: XREAL SDK 3.x** (package `com.xreal.xr`, built on Unity XR Plug-in Management, XR Interaction Toolkit,
AR Foundation and XR Hands). NRSDK is legacy: `install_xreal_sdk` installs the SDK 3.x tarball, the old NRSDK import
tool is gone, and `validate_xreal_setup` only warns if NRSDK files are still in the project. The migration is in
progress; the remaining work is in [ROADMAP.md](ROADMAP.md).

**Works today** (run in a 6000.6.4f1 Editor on 2026-10-11, not yet on a device): `install_xreal_sdk` (validates and
adds the SDK tarball), `setup_xreal_project` and
`validate_xreal_setup` (XREAL's Getting Started player settings, XREAL loader assignment, XR Project Validation),
`configure_xreal_settings`, `set_tracking_mode` and `configure_xreal_display` (XREALSettings: tracking type, input
source, stereo mode, multi-resume; XR camera background), `enable_hand_tracking` (device-gated; XREAL's own hand
tracking setup menu), `setup_xr_interaction`, `create_xr_rig`, `add_xr_interactor`, `create_xr_ui`,
`create_hand_interactable` (through the XR Interaction Toolkit) and `enable_plane_detection`
(through AR Foundation). Partial: `build_xreal_apk` with `get_build_status`, `get_connected_devices`,
`configure_android_build`, `create_spatial_anchor`, `profile_xr_scene`.

**Removed** because the SDK has no such feature: `calibrate_glasses`, `configure_hand_gestures`,
`configure_passthrough`, `configure_occlusion` and the AR / VR / MR modes of the old `set_render_mode`.

**Not there yet:** spatial anchor persistence, image tracking, meshing, tracking-mode switching at runtime and every
live read of device state (hands, planes, images, glasses status), which need a device or a running XR subsystem.
Those tools return "not implemented" or "not available".

**One Pro feature limits.** XREAL's docs for SDK 3.1.0 list the *One Series*; they do not name the One Pro
separately, so it is treated as part of that series.

| Feature | XREAL One / One Pro | Air 2 Ultra |
|---|---|---|
| 6DoF head tracking | Only with the XREAL Eye accessory and updated firmware; otherwise 3DoF | Yes |
| Hand tracking | No | Yes |
| Plane detection | No | Yes |
| Image tracking | No | Yes |
| Spatial anchors | No | Yes |
| Depth mesh | No | Yes |
| RGB camera | Only with the Eye accessory | n/a |

The hand, plane, image, anchor and meshing tools therefore apply to an Air 2 Ultra, not a One Pro. Only
`enable_hand_tracking` is gated by device so far (it refuses One / One Pro unless `force: true`); the others do not
check, because the SDK itself does not.

**Requirements and cautions**
- Unity: XREAL documents 2021.3, 2022.3 and 6000.0. This plugin ran live on 6000.5 and is compile-checked on 6000.5
  and 6000.6, but XREAL SDK 3.x on 6000.6 is unverified.
- Host: XREAL Beam Pro, or a Samsung Galaxy S24 or S25 with the ControlGlasses app. XREAL warns against upgrading
  development hosts to Android 16.
- The SDK tarball (`com.xreal.xr.tar.gz`) comes from [developer.xreal.com/download](https://developer.xreal.com/download/)
  behind XREAL's terms. `install_xreal_sdk` installs it from a local path you supply (`tarballPath`; `setup_xreal_project`
  accepts the same through `xrealSdkPath`); this repo never downloads or bundles it.
- XREAL SDK 3.x has no simulator of its own. Use the XR Interaction Toolkit's XR Device Simulator (head pose and
  controllers only).
- The first run of the XR tools in a real project is still ahead; the behaviour-test plan is in
  [`docs/HANDLER_AUDIT.md`](docs/HANDLER_AUDIT.md) section 11.

Sources: [Getting Started](https://docs.xreal.com/Getting%20Started%20with%20XREAL%20SDK),
[device compatibility](https://docs.xreal.com/XREALDevices/Compatibility),
[migration from NRSDK](https://docs.xreal.com/MigratingFromNRSDKToXREALSDK/intro).

---

## Relationship to Unity's official tooling

Unity now ships its own MCP entry point: **`unity mcp`**, part of the Unity CLI (beta). It starts an MCP server over
STDIO that exposes the commands of Unity's **Pipeline** package (`com.unity.pipeline`, experimental; Unity 6000.0+) as
tools, about 160 of them, free of charge. Unity has deprecated the MCP server inside the AI Assistant package
(`com.unity.ai.assistant`) in its favour
([Unity's migration page](https://docs.unity.com/en-us/unity-cli/replace-mcp-server-unity-cli)). These details
were checked on 2026-10-10 and will move while the CLI and package are in beta.

Treat it as complementary to this server:
- Pipeline covers core Editor control well: scenes, GameObjects, components, prefabs, assets, tests, builds, baking,
  packages, console, screenshots and transactional batches. On Unity 6.0+ projects with Pipeline installed, prefer it for
  that; the overlapping tools here are not being extended (see [ROADMAP.md](ROADMAP.md)).
- This server covers what Pipeline does not: XREAL and XR tooling, terrain, tilemap, sprites, particles, physics
  queries, audio mixer, profiler captures, Asset Store, scripted play-mode drives (`play_sequence`, `playtest`),
  script management, the debugger, knowledge search, and MCP resources and prompts (Pipeline serves tools only).
  It also runs on Unity versions before 6.0, which XREAL's docs still list.
- You can register both servers in one client. Together they advertise roughly 245 tools, so consider tool filtering.
- Try it: `unity pipeline install` in a project, then `unity mcp --project-path <path>`.

---

## Installation

You need the Node server and the Unity plugin.

```bash
npm install
npm run build
```

Add the plugin to a Unity project, either by copying `UnityPlugin/` to `Packages/com.joelwehr.unity-mcp/` or by adding
this to `Packages/manifest.json`:

```json
"com.joelwehr.unity-mcp": "file:../path/to/unity-mcp/UnityPlugin"
```

The plugin depends on `com.unity.ugui` (declared in its package manifest). It starts its bridge when the Editor
opens; **Tools > Unity MCP** has Settings, Start, Stop and Restart. Step-by-step setup, including XREAL, is in
[SETUP_GUIDE.md](SETUP_GUIDE.md).

## Configuration

The MCP client launches the Node server; the server connects to the bridge in the Editor. Nothing listens on
8090 except Unity.

### Claude Code (project-level)

Create `.mcp.json` in your project (the one checked in here has a machine-specific path; edit it):

```json
{
  "mcpServers": {
    "unity-mcp": {
      "command": "node",
      "args": ["C:/path/to/unity-mcp/build/index.js"],
      "env": {
        "UNITY_PORT": "8090",
        "UNITY_HOST": "localhost",
        "LOGGING": "true"
      }
    }
  }
}
```

### Claude Desktop

Add to `%APPDATA%/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "unity-mcp": {
      "command": "node",
      "args": ["C:/path/to/unity-mcp/build/index.js"],
      "env": {
        "UNITY_PORT": "8090"
      }
    }
  }
}
```

### With RAG knowledge search

`search_unity_knowledge` needs an external RAG server; without these variables it replies that search is not configured.

```json
{
  "mcpServers": {
    "unity-mcp": {
      "command": "node",
      "args": ["path/to/unity-mcp/build/index.js"],
      "env": {
        "UNITY_PORT": "8090",
        "RAG_PYTHON_PATH": "python",
        "RAG_SERVER_PATH": "path/to/unity-rag-server",
        "RAG_DB_PATH": "path/to/unity-rag-server/data"
      }
    }
  }
}
```

## Environment variables

| Variable | Description | Default |
|----------|-------------|---------|
| `UNITY_PORT` | Port of the Unity bridge (falls back to `Port` in `ProjectSettings/McpUnitySettings.json`) | `8090` |
| `UNITY_HOST` | Host the Node server connects to. Keep it local: the bridge listens on loopback only | `localhost` |
| `LOGGING` | Set to `true` to enable console logging | off |
| `LOGGING_FILE` | Set to `true` to enable file logging | off |
| `RAG_PYTHON_PATH` | Path to the Python executable for knowledge search | - |
| `RAG_SERVER_PATH` | Path to the RAG server directory | - |
| `RAG_DB_PATH` | Path to the RAG database | - |

### Unity command-line flags

The plugin auto-starts its WebSocket server in the interactive Editor (unless *Auto Start* is
off in **Tools > Unity MCP > Settings**). In `-batchmode` it does **not** start, so headless builds
and test runs on other checkouts don't fight the Editor for the port.

| Flag | Effect |
|------|--------|
| `-mcpServer` | Start the bridge anyway (required to use it from `-batchmode`) |
| `-mcpPort <port>` | Listen on this port instead of the configured one (e.g. `-batchmode -mcpServer -mcpPort 8091`) |

If the port is already taken, the plugin logs one warning per Editor session and stays stopped.

---

## Quick start: XREAL

Use these in order. Several steps install packages and trigger a script reload; the tools return `pending: true` and
you call them again afterwards. Details and caveats are in [SETUP_GUIDE.md](SETUP_GUIDE.md).

```
1. setup_xr_interaction  { interactionMode: "Controller" }
2. setup_xreal_project   { xrealSdkPath: "C:/path/to/com.xreal.xr.tar.gz" }   // call again after the reload to assign the loader
3. validate_xreal_setup  { autoFix: false }
4. create_xr_rig
5. create_hand_interactable { targetGameObject: "MyCube", interactionType: "Grab" }
6. build_xreal_apk       { buildType: "Development" }                         // then poll get_build_status
```

`interactionMode: "Controller"` avoids the XR Hands package, which only matters on an Air 2 Ultra.

---

## Architecture

```
+------------------+   STDIO   +------------------+   WebSocket 127.0.0.1:8090   +------------------+
|  AI assistant    | <-------> |  Unity MCP       | <--------------------------> |  Unity Editor    |
|  (MCP client)    |           |  server (Node)   |                              |  + C# plugin     |
+------------------+           +------------------+                              +--------+---------+
                                                                                          |
                                                          adb / USB (XREAL builds only)   |
                                                                                          v
                                                                   +--------------------------------------+
                                                                   | Android host (Beam Pro, Galaxy S24/25)|
                                                                   | + XREAL glasses                      |
                                                                   +--------------------------------------+
```

## Development

```bash
npm run build          # compile TypeScript to build/
npm run watch          # compile in watch mode
npx tsc --noEmit       # typecheck only (this is what CI runs, plus the build)
npm run inspector      # run with the MCP Inspector
npm start              # run the server (the MCP client normally does this)
npm run compile-check  # compile the C# plugin offline, without launching Unity
```

### `npm run compile-check`

`scripts/compile-check.mjs` compiles every asmdef under `UnityPlugin/` with the .NET SDK's Roslyn compiler against
the reference assemblies of an installed Unity editor. It never starts Unity or contacts a running Editor.

```bash
npm run compile-check                              # newest 6000.6.x under C:/Program Files/Unity/Hub/Editor
npm run compile-check -- --editor 6000.5 --warnings
npm run compile-check -- --packages path/to/Project/Library/ScriptAssemblies
```

- Needs Node 20+ and a .NET SDK on `PATH` (`dotnet --list-sdks`).
- Package DLLs (uGUI, Input System, ...) are read from a project's `Library/ScriptAssemblies`; point at one with
  `--packages` or `UNITY_MCP_PACKAGE_DLLS`. XR package DLLs are normally absent, so the XR code (reflection only) is
  compiled but its member names are not checked.
- Prints `N errors, M warnings` and the first errors. Exit code 0 = no errors, 1 = compile errors, 2 = environment
  problem. Other options: `--unity-root`, `--plugin`, `--define`, `--max-errors`, `--json`, `--keep`.
- It is not Unity's compile pipeline: no analyzers, no asmdef GUID resolution, and the `#if` branches for editors you
  do not have installed (such as 2022.3) are not compiled.

## File structure

```
unity-mcp/
├── src/
│   ├── index.ts                    # Server entry point, registers everything
│   ├── unity/mcpUnity.ts           # WebSocket client for the Unity bridge
│   ├── tools/                      # Tool implementations (52 core)
│   │   └── xreal/                  # XREAL tools (30)
│   ├── resources/                  # Resources (11 core)
│   │   └── xreal/                  # XREAL resources (6)
│   ├── prompts/                    # Prompts (9 core)
│   │   └── xreal/                  # XREAL prompts (4)
│   └── utils/                      # logger, errors, unityResult, progress, annotations, tarball
├── UnityPlugin/                    # Unity package com.joelwehr.unity-mcp
│   ├── Editor/Core/                # WebSocket server, dispatch, settings
│   ├── Editor/Handlers/            # C# handlers (Xreal/ for XREAL and XR)
│   └── Runtime/                    # Play-mode runner for play_sequence
├── scripts/compile-check.mjs       # Offline C# compile check
├── docs/HANDLER_AUDIT.md           # Handler audit and fix plan
├── build/                          # Compiled JavaScript
├── .mcp.json                       # Sample MCP client configuration
└── package.json
```

## Summary

| Category | Count |
|----------|-------|
| Core tools | 52 |
| XREAL tools | 30 |
| **Total tools** | **82** |
| Core resources | 11 |
| XREAL resources | 6 |
| **Total resources** | **17** |
| Core prompts | 9 |
| XREAL prompts | 4 |
| **Total prompts** | **13** |

---

## License

MIT