maige-3d-mcp
Enables programmatic control and manipulation of A-Frame scenes, including entity creation and scene management.
Provides tools for controlling Babylon.js scenes, allowing for mesh updates, lighting configuration, and camera movements.
Supports the creation and import of 3D models using the glTF format within the managed 3D scenes.
Enables direct in-world AI chat functionality within 3D environments by integrating with OpenAI's models.
Supports controlling React Three Fiber (R3F) scenes, enabling AI-driven updates to 3D components within React-based environments.
Allows control of live Three.js scenes, including tools for managing objects, lighting, cameras, animations, and environment settings.
Targets high-performance 3D rendering and scene manipulation within WebGPU-based environments.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@maige-3d-mcp Add a spinning blue sphere and set the background to a sunset orange."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
m{ai}geXR-3d-mcp
The multi-framework 3D MCP server that actually works. Control live Three.js, A-Frame, Babylon.js, and React Three Fiber scenes from any MCP-capable AI — GitHub Copilot, Claude Desktop, Cursor, you name it — with in-world async chat so you can talk to the AI from inside the 3D canvas while it reshapes the world around you.
All four framework clients render matching, visually-aligned output from a single unified scene state. Same prompt, same scene, any engine.
Highlights
4 production-ready 3D clients — Three.js, A-Frame (1.7.0 + bloom), Babylon.js (PBR), React Three Fiber + Zustand — all visually aligned
WebXR / VR support — enter immersive VR in all four clients; floating chat panel follows your gaze so you can talk to the AI from inside the headset
9 AI providers out of the box — OpenAI (GPT-5.2), Anthropic (Claude Sonnet 4.6), Google Gemini 3.1 Pro, Mistral, Groq (Llama 3.3 70B), xAI Grok-4, Cohere Command R+, Together.ai, and local Ollama
23 MCP tools — objects, lights, cameras, animation, environment, scene I/O, undo/redo, screenshots, and in-world chat
Persistent animations — animations survive page reloads; the server stores active animations in scene state and replays them when clients reconnect
Concurrent animations — multiple properties (position, rotation, scale) animate simultaneously on the same object across all engines
Per-framework system prompts — each client tells the AI how to generate geometries, materials, and lighting that look correct in that engine (adapted from the iOS maigeXR app)
In-world chat — press
~to talk to the AI without leaving the 3D viewport; it reads your messages and answers in a floating overlayScene-aware AI — 20-turn conversation history + live scene state injection ensures the AI makes incremental edits, not destructive rebuilds
One command —
pnpm devstarts the server + all four clients simultaneously; auto-opens the Three.js client in your browserHot-swappable AI provider — change provider mid-session from the client dropdown; no restart needed
Related MCP server: threejs-devtools-mcp
Quick start
Option A — npx (no clone needed)
npx maige-3d-mcpThis starts the MCP server via stdio. Point your MCP client (VS Code Copilot, Claude Desktop, Cursor) at the command npx maige-3d-mcp.
Option B — from source
1. Install
cd mcp-webgpu
pnpm install2. Configure
cp .env.example .envAdd at least one API key. All variables:
Variable | Default | Purpose |
|
| WebSocket bridge port |
|
| Active AI provider ( |
| — | OpenAI |
| — | Anthropic |
| — | Google Gemini |
| — | Mistral |
| — | Groq |
| — | xAI / Grok |
| — | Cohere |
| — | Together.ai |
|
| Local Ollama |
|
| Open browser on startup |
|
| Which client to auto-open ( |
Each provider also has a *_MODEL env var (e.g. OPENAI_MODEL=gpt-4.1) — see .env.example for all available models.
Two chat modes:
Relay mode (no key needed): the MCP host AI (Copilot, Claude Desktop) handles in-world chat by polling
getPendingUserMessages.Direct mode (key in
.env): the server answers chat autonomously using the configured provider.
3. Build & run
pnpm build:server # compile TypeScript once
pnpm dev # start server + all 4 clientsClients open at:
Framework | URL |
Three.js | |
A-Frame | |
Babylon.js | |
React Three Fiber |
4. Register with VS Code Copilot
The .vscode/mcp.json is pre-configured. Reload VS Code and maige-3d-mcp appears in Copilot agent mode.
Alternatively, add to your global VS Code settings:
// .vscode/mcp.json (already included)
{
"servers": {
"maige-3d-mcp": {
"type": "stdio",
"command": "node",
"args": ["packages/server/build/main.js"],
"env": {
"OPENAI_API_KEY": "${env:OPENAI_API_KEY}",
"ANTHROPIC_API_KEY": "${env:ANTHROPIC_API_KEY}",
"GOOGLE_API_KEY": "${env:GOOGLE_API_KEY}"
}
}
}
}Supported AI Providers (9)
Provider | Default Model | Available Models | Notes |
OpenAI |
| gpt-5.2, gpt-5.2-pro, gpt-4.1, gpt-4.1-mini, gpt-4o, o3, o4-mini | Best general-purpose option |
Anthropic |
| claude-opus-4-6, claude-sonnet-4-6, claude-sonnet-4-5, claude-haiku-4-5 | Strong reasoning |
Google Gemini |
| gemini-3.1-pro-preview, gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite | Latest flagship, multimodal |
Mistral |
| mistral-large, mistral-medium, mistral-small, open-mistral-nemo | Fast + capable |
Groq |
| llama-3.3-70b, deepseek-r1-distill-llama-70b, llama-3.1-8b-instant, mixtral-8x7b | Blazing inference speed |
xAI / Grok |
| grok-4-0709, grok-4-fast-reasoning, grok-3, grok-3-mini, grok-code-fast-1 | Latest Grok 4 |
Cohere |
| command-r-plus, command-r, command, command-light | Tool-use focused |
Together.ai |
| DeepSeek-R1-Distill-70B-free, Llama-3.3-70B-free, Llama-3.3-70B, DeepSeek-R1, Qwen2.5-72B | Free tier available |
Ollama |
| llama3.2, mistral, phi4, gemma3, qwen2.5, deepseek-r1 | Fully local, no API key |
Switch providers from the dropdown in the chat overlay or by changing CHAT_PROVIDER in .env. Override the model per-provider with *_MODEL env vars (see .env.example).
Available MCP Tools (23)
Objects
Tool | Description |
| Add a mesh — box, sphere, cylinder, cone, torus, plane, capsule, or glTF model |
| Partial update: position, rotation, scale, material (color, metalness, roughness, emissive), visibility |
| Remove by id |
| Duplicate with optional offset |
| Inspect a single object |
| Full scene JSON snapshot |
Lights
createLight · updateLight · deleteLight — ambient, directional, point, spot, hemisphere
Camera
setCamera · flyToObject
Animation
animateObject · stopAnimation — rotate, bounce, pulse, float, spin, custom keyframes. Animations persist in server state and replay automatically on page reload. Multiple properties animate concurrently per object.
Environment
setEnvironment — background color, fog, tone mapping, exposure, shadow toggle
Scene
clearScene · loadScene · exportScene · undo · redo · takeScreenshot
In-world Chat
Tool | Description |
| Retrieve messages typed from inside the 3D canvas |
| Display AI reply in the floating overlay |
| Flush the queue |
MCP Resources
URI | Description |
| Live JSON snapshot of all objects, lights, camera, and environment |
| List of currently connected browser sessions (id, framework, timestamp) |
MCP Prompts
Prompt | Description |
| Full system context for AI assistants — scene tools, chat workflow, incremental update rules, 10+ advanced demo recipes (galaxy, DNA helix, neon tunnel, crystal cluster, etc.) |
| Per-framework geometry/material/lighting tips. Accepts |
| NEW: Instant access to 10+ stunning pre-built demo templates. Perfect for quick impressive visualizations. Accepts |
WebXR / VR Support
All four clients support immersive VR via WebXR. Click the 🥽 Enter VR button (bottom-left) to start a session.
Framework | Implementation |
Three.js | Custom |
A-Frame | Native |
Babylon.js |
|
React Three Fiber |
|
In VR, the chat panel floats in front of you and follows your gaze. AI replies appear in real-time so you can direct the scene from inside the headset.
Requires a WebXR-capable browser (Chrome 79+, Edge 79+, Meta Quest Browser) and a VR headset or the WebXR API Emulator extension for desktop testing.
In-world Chat
Press ~ (backtick) or click AI Chat in the bottom-right corner. Type a message, hit Enter, and the AI receives it, acts on it, and replies — all without leaving the 3D viewport. The chat overlay also includes:
Provider selector — switch AI providers on the fly
System prompt editor — customise the AI's behaviour per session
Clear Scene button — reset the world instantly
Debug panel — press Escape to inspect scene state and connection info
Architecture
┌─────────────────────────┐
│ MCP Host (Copilot, │ stdio / JSON-RPC
│ Claude, Cursor, etc.) │◄────────────────────┐
└─────────────────────────┘ │
│
┌──────────────────┴──────────────────┐
│ MCP Server (Node.js) │
│ │
│ tools/ ─ 23 tool definitions │
│ state/ ─ SceneStateManager + Undo │
│ chat/ ─ ChatRelay (9 providers) │
│ ws/ ─ WebSocket bridge :8083 │
│ └── adapters/ (per-framework) │
└──────────┬───────────────────────────┘
│ WebSocket
┌────────────────┬───────────────┼───────────────┬────────────────┐
▼ ▼ ▼ ▼ │
┌──────────────┐ ┌──────────────┐ ┌───────────────┐ ┌──────────────┐ │
│ Three.js │ │ A-Frame │ │ Babylon.js │ │ R3F / React │ │
│ :5173 │ │ :5174 │ │ :5175 │ │ :5176 │ │
│ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │
│ │VR/WebXR │ │ │ │VR/WebXR │ │ │ │VR/WebXR │ │ │ │VR/WebXR │ │ │
│ │ChatPanel │ │ │ │ChatPanel │ │ │ │ChatPanel │ │ │ │ChatPanel │ │ │
│ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │
└──────────────┘ └──────────────┘ └───────────────┘ └──────────────┘Each client connects via WebSocket to the same MCP server. The server maintains a single canonical scene state and pushes commands through per-framework adapters that translate Vec3 formats, material models, and geometry names into each engine's native representation.
Key server features:
Conversation history — the AI remembers the last 20 turns of dialogue, avoiding destructive scene rebuilds
Scene state injection — every AI call includes a summary of current objects, lights, and environment so the AI knows what already exists
Per-framework system prompts — each client tells the AI how to generate geometries, materials, and lighting that look correct in that specific engine
Undo/redo — 20-deep snapshot stack on the server, triggered via MCP tools
Animation persistence — active animations are stored in scene state and replayed on reconnect, so looping animations survive page reloads
Project Layout
mcp-webgpu/
├── .env.example ← all env vars + model lists documented
├── .vscode/mcp.json ← pre-configured for VS Code Copilot agent mode
├── package.json ← pnpm workspace root
├── PLAN.md ← full architecture plan
├── packages/
│ ├── server/ ← MCP server (TypeScript / Node)
│ │ └── src/
│ │ ├── main.ts ← entry + .env discovery
│ │ ├── types.ts ← shared types
│ │ ├── tools/ ← 23 MCP tool definitions
│ │ ├── handlers/ ← tool / prompt / resource handlers
│ │ ├── state/ ← SceneStateManager + UndoStack
│ │ ├── chat/ ← ChatRelay (9 providers) + MessageQueue
│ │ └── ws/ ← WebSocket server + framework adapters
│ │ └── adapters/ ← ThreeAdapter, AFrameAdapter,
│ │ BabylonAdapter, R3FAdapter
│ ├── client-threejs/ ← Three.js (Vite)
│ │ └── src/
│ │ ├── scene.ts ← SceneManager
│ │ ├── commands/ ← command dispatcher
│ │ ├── overlay/ ← ChatOverlay UI
│ │ └── vr/ ← VRSetup + VRChatPanel (WebXR)
│ ├── client-aframe/ ← A-Frame 1.7.0 + bloom (Vite)
│ │ └── src/
│ │ ├── scene.ts ← A-Frame SceneManager
│ │ ├── commands/ ← command dispatcher
│ │ └── overlay/ ← ChatOverlay UI
│ ├── client-babylonjs/ ← Babylon.js + PBR (Vite)
│ │ └── src/
│ │ ├── scene.ts ← Babylon SceneManager
│ │ ├── commands/ ← command dispatcher
│ │ └── overlay/ ← ChatOverlay UI
│ └── client-r3f/ ← React Three Fiber + Zustand (Vite)
│ └── src/
│ ├── App.tsx ← React app shell
│ ├── SceneCanvas.tsx ← R3F canvas + XR wrapper
│ ├── store/ ← Zustand scene store
│ ├── commands/ ← command dispatcher
│ ├── overlay/ ← ChatOverlay UI
│ └── vr/ ← VRChatPanel (React XR component)Roadmap
Phase 1 — Three.js client + full tool set + in-world chat
Phase 2 — A-Frame client (1.7.0, bloom post-processing) + Babylon.js client (PBR materials)
Phase 3 — React Three Fiber client (Zustand state, drei helpers)
Phase 3.5 — 9 AI providers, per-framework system prompts, visual alignment across all 4 engines
Phase 4 — WebXR / VR headset support (all 4 clients + floating VR chat panel)
Phase 5 — VS Code MCP config, auto-open browser, conversation history + scene state awareness
Phase 6 — Animation persistence, concurrent multi-property animations, enhanced default lighting (ambient + hemisphere + directional with shadows)
License
MIT
Available Tools
33 toolsaddBehaviorA
Attach a continuous frame-tick behavior to an object. Behaviors run every frame until removed. Types: spin (rotate continuously), bob (oscillate up/down), orbit (circle around a point), lookAt (always face camera), pulse (rhythmic scale breathing).
Inputs: id (required), objectId (required), type (required), params.
animateObjectA
Animate an object property to a target value. Supports transforms (position, rotation, scale) and material properties (emissiveIntensity, opacity, color) for pulsing glow, fading, color shifts.
Inputs: id (required), property (required), to (required), duration, +2 more.
clearPendingMessagesA
Clear all queued in-world user messages (useful after processing a batch).
Inputs: no named parameters.
clearSceneA
Remove all user-created objects from the scene and reset default lighting.
Inputs: no named parameters.
cloneObjectC
Duplicate an object with an optional position offset.
Inputs: id (required), newId, offset.
createLightC
Add a light to the scene.
Inputs: lightType (required), id, color, intensity, +7 more.
createObjectA
Add a 3D object to the live scene. Returns the assigned id. Always call getSceneState first to understand what already exists.
Inputs: type (required), id, position, rotation, +12 more.
createParticlesA
Create a particle system (stars, dust, sparks, snow, fire embers, laser trails). Uses GPU-efficient point sprites with optional drift animation and twinkle.
Inputs: count (required), spread (required), id, position, +10 more.
deleteLightC
Remove a light from the scene by id.
Inputs: id (required).
deleteObjectC
Remove a 3D object from the scene by id.
Inputs: id (required).
deleteParticlesC
Remove a particle system from the scene.
Inputs: id (required).
executeScriptA
Execute arbitrary JavaScript code in the connected browser scene context. The code runs inside an async function with access to: scene (THREE.Scene), camera, renderer, controls, and a helpers object. Return a value to send it back. Use this for advanced effects the other tools cannot achieve — custom shaders, complex geometry, procedural generation, physics, etc.
Inputs: code (required).
exportSceneA
Export the current scene as a JSON string (can be stored and re-loaded later).
Inputs: no named parameters.
exportStandaloneSceneA
Export the current scene as a standalone HTML file that plays in any browser without a server. Includes all objects, lights, materials, animations, camera, and environment. No chat UI. Saved to the scenes/ folder. Returns the file path.
Inputs: name.
flyToObjectB
Smoothly move the camera to look at an object.
Inputs: id (required), distance, duration.
getObjectC
Get properties of a single scene object by id.
Inputs: id (required).
getPendingUserMessagesA
Return any in-world chat messages the user has typed from inside the 3D environment. Call this at the start of every turn to check for user input from the canvas.
Inputs: no named parameters.
getSceneStateA
Get the full current scene state: all objects, lights, camera, and environment. Call this first before making any changes.
Inputs: no named parameters.
listScenesB
List all saved scene JSON files in the scenes/ folder.
Inputs: no named parameters.
loadSceneB
Replace the entire scene with a previously exported scene JSON.
Inputs: sceneJson (required).
loadSceneFromFileC
Load a previously saved scene from the scenes/ folder by name.
Inputs: name (required).
redoB
Redo a previously undone scene change.
Inputs: no named parameters.
removeBehaviorC
Remove a behavior by its ID, stopping its frame-tick effect.
Inputs: id (required).
saveSceneB
Save the current scene to a JSON file in the scenes/ folder. Returns the file path.
Inputs: name.
sendChatMessageA
Display a message in the in-world chat overlay visible to the user inside the 3D view.
Inputs: message (required), sessionId.
setCameraC
Set the camera position and look-at target.
Inputs: position, target, fov.
setEnvironmentC
Configure the scene environment: background, fog, tone mapping, shadows, and post-processing effects.
Inputs: background, hdriUrl, fog, shadows, +5 more.
stopAnimationC
Stop any running animation on an object.
Inputs: id (required).
takeScreenshotA
Capture the current 3D viewport and return a base-64 PNG data URL. Requires a browser client to be connected.
Inputs: no named parameters.
undoB
Undo the last scene change (up to 20 levels).
Inputs: no named parameters.
updateLightC
Update an existing light by id.
Inputs: id (required), color, intensity, position, +1 more.
updateObjectC
Update any properties of an existing 3D object. Supports partial updates.
Inputs: id (required), position, rotation, scale, +2 more.
updateParticlesC
Update properties of an existing particle system.
Inputs: id (required), position, color, size, +4 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 33 tools
Most tools have distinct purposes with clear boundaries, such as createObject vs. updateObject, or animateObject vs. addBehavior. However, some overlap exists: animateObject and addBehavior both handle object animations, which could cause confusion in selection, though their descriptions clarify animateObject is for property transitions while addBehavior is for continuous frame-tick effects.
Tool names follow a highly consistent verb_noun pattern throughout, such as createObject, deleteObject, updateObject, and getSceneState. All tools use camelCase uniformly, with no mixing of naming conventions, making the set predictable and easy to navigate.
With 33 tools, the count is borderline high for a 3D scene management server, potentially overwhelming for agents. While the tools cover a comprehensive range of operations, the number may feel heavy compared to typical well-scoped servers, though it aligns with the domain's complexity.
The tool set provides complete coverage for 3D scene management, including CRUD operations for objects, lights, and particles, scene state management, animation controls, user interaction handling, and advanced features like scripting and export. No obvious gaps exist; agents can perform full lifecycle workflows without dead ends.
Maintenance
Related MCP Connectors
Build, edit and publish live 3D, AR and game scenes in ARClip from your AI assistant.
Generate 3D models, worlds, PBR materials, and coordinated asset packs from AI agents.
3D avatars, embeds, glTF tools, agent memory, and on-chain agent identity from three.ws.
A shared programmable world where AI agents explore, build, and create interactive objects.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to inspect and modify live Three.js scenes in real time through 59 tools covering objects, materials, shaders, textures, animations, performance monitoring, memory diagnostics, and code generation. Works with vanilla Three.js, React Three Fiber, and any framework without changing your project, using an injected browser bridge and in-page devtools overlay.5,671 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to inspect and edit Three.js scenes, materials, shaders, lights, animations, and performance in real time, preview changes live in the browser, capture frames, and turn approved changes into code. It supports vanilla Three.js and React Three Fiber across WebGL/WebGPU without modifying the project.5,671 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to inspect and edit live Three.js scenes in the browser — objects, materials, shaders, textures, lights, animation, performance and memory — with 69 tools covering scene trees, screenshots, website audits and video frame capture. Changes can be previewed live with zero project modifications, then converted into ready-to-paste vanilla Three.js or React Three Fiber code.5,671 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to inspect, debug and edit Three.js scenes in real time from a live browser, covering objects, materials, shaders, textures, lights, animation, and performance. Live preview changes can be audited, reverted, or turned into ready-to-paste vanilla Three.js and React Three Fiber code without modifying the project.5,671 npmMIT