webots-mcp

Give AI agents full control and observability of the Webots robot simulator.

An MCP (Model Context Protocol) server that lets any MCP client — Claude Code, Codex, opencode, or your own agent — launch simulations, train reinforcement-learning models, watch what the robot is doing while it trains (without slowing training down), evaluate trained models, and manipulate the scene interactively.

Ask your agent things like:

"Start a 200k-step training run and tell me how it's going every few minutes." "Why did the reward drop? Show me a screenshot of what the robot is doing right now." "Evaluate the latest model for 10 episodes and compare it with yesterday's run." "Freeze the sim, put a box 25 cm in front of the robot, step 100 timesteps, and show me the sensor readings."

How it works

The Webots simulation state is only reachable from a controller inside the simulation loop — and during training, the robot's controller is the RL trainer. This server solves that with two independent channels that never touch the training hot path:

┌────────────── AI agent (any MCP client) ──────────────┐
│                 webots-mcp  (server.py)                │
└──────┬──────────────────┬──────────────────┬──────────┘
       │ JSON-TCP         │ files            │ subprocess
       ▼                  ▼                  ▼
  OBSERVER robot     runs/*/metrics.jsonl   webots / training
  (webots_bridge.py, (written by the SB3    (start, stop,
   2nd supervisor,    callback in           monitor)
   <extern>)          rl_metrics.py)
       │                  ▲
       ▼                  │
    Webots ◄────── training controller (untouched)

1. Observer robot — an invisible second supervisor robot (synchronization FALSE) runs webots_bridge.py as an extern controller. It reads pose, velocity, scene objects and takes screenshots in parallel with training, costing ~3% fps.

2. Metrics files — a Stable-Baselines3 callback (rl_metrics.py) appends one JSON line per episode (with per-component reward breakdown and termination reason) and per policy update (fps, losses). The server only reads files: training never waits for the agent.

3. Process management — the server launches and stops Webots instances as subprocesses, each on its own port.

All state lives in files — killing the server loses nothing.

Quick start

Requirements: Python ≥ 3.10 and Webots (tested with R2025a on macOS; Linux/Windows install layouts are auto-detected via WEBOTS_HOME or PATH).

pip install git+https://github.com/bazucas/Webots_MCP        # installs the `webots-mcp` command
# with the RL extras (stable-baselines3 + gymnasium, for training/eval):
pip install "webots-mcp[rl] @ git+https://github.com/bazucas/Webots_MCP"

Or without installing anything, via uv:

uvx --from git+https://github.com/bazucas/Webots_MCP webots-mcp

Or clone and run from source: git clone ... && pip install -e ".[dev]".

Then register the server in your MCP client — the working directory must be your Webots project root (that's how the server finds worlds and runs/):

Claude Code

.mcp.json in the project root (cwd is the project automatically):

{
  "mcpServers": {
    "webots": { "command": "webots-mcp" }
  }
}

(If you didn't pip install, use "command": "python3", "args": ["/path/to/webots_mcp/webots_mcp/server.py"] — same for the clients below.)

opencode

opencode.json in the project root:

{
  "mcp": {
    "webots": { "type": "local", "command": ["webots-mcp"] }
  }
}

OpenAI Codex CLI (GPT)

~/.codex/config.toml (set WEBOTS_MCP_PROJECT since cwd may vary):

[mcp_servers.webots]
command = "webots-mcp"
env = { WEBOTS_MCP_PROJECT = "/home/you/my-webots-project" }

Cursor

.cursor/mcp.json in the project root — same shape as Claude Code's .mcp.json.

Claude Desktop / other MCP clients

Any client that speaks MCP over stdio works: command webots-mcp (or python3 /path/to/webots_mcp/webots_mcp/server.py). If the client doesn't launch it from your project directory (Claude Desktop doesn't), set WEBOTS_MCP_PROJECT in the server's env.

Restart/reconnect the client and approve the server — 23 tools appear.

Configuration

Everything is overridable with environment variables:

Tools

Process management

Training

Live observation (during training)

Evaluation

Interactive control (world with the robot's controller set to <extern>)

Plus the MCP prompt diagnose_training(run_id?) — a guided training-health analysis (trend, terminations, reward hacking signals, concrete shaping suggestions) — and resources webots://runs and webots://runs/{run_id}/metrics.

Adapting it to your project

The defaults reflect the project this was born in (Thymio II, ISCTE-IUL robotics course), but the pattern is generic. Your project needs:

1. An observer robot in the training world (this is the whole trick — copy-paste):

Robot {
  name "observer"
  supervisor TRUE
  synchronization FALSE
  controller "<extern>"
}

2. A DEF ROBOT on your main robot node, so the bridge can find it from outside.

3. The metrics callback in your SB3 training script:

from rl_metrics import MetricsCallback
model.learn(total_timesteps=n, callback=MetricsCallback(run_name="ppo", runs_root="/path/to/project/runs"))

For per-component reward breakdowns, have your env put them in the info dict at episode end:

info["episode_metrics"] = {
    "termination_reason": "fall",           # your labels
    "steps": steps, "distance": d, "cells_visited": c,
    "reward_components": {"forward": 12.3, "cliff": -5.0},
}

4. For train_start, your training controller should read TRAIN_TIMESTEPS from the environment; for eval_run, your evaluation controller should read EVAL_ALGO, EVAL_EPISODES, EVAL_DETERMINISTIC and write results to the JSON path in EVAL_OUT.

webots_bridge.py also works standalone without MCP — one JSON command per TCP connection on 127.0.0.1:10101:

printf '{"cmd":"get_obs"}\n' | nc -w 5 127.0.0.1 10101

Tests

pip install -e ".[dev]"
pytest tests -m "not webots"   # unit + MCP stdio protocol (~1 s, no Webots needed)
pytest tests -m webots         # integration against a real Webots (~20 s)
pytest tests                   # everything (37 tests)

CI (GitHub Actions) runs the fast suite on Linux, macOS and Windows for every push.

Integration tests need a real Webots project (WEBOTS_MCP_PROJECT); they skip cleanly if it's missing. Bridge unit tests inject a fake Webots API via sys.modules["controller"], so most of the suite runs anywhere.

Limitations & notes

• Webots itself was only exercised on macOS (R2025a). The server code runs on Linux/Windows in CI, and install layouts for all three OSes are covered by unit tests, but end-to-end Webots runs on Linux/Windows are untested — PRs welcome.

• The bridge listens on localhost only, with no authentication — don't expose the port.

• One bridge connection at a time (BRIDGE_PORT, default 10101).

• Avoid simulationReset in training code (it kills the observer's connection); reset by teleporting the robot instead.

License

MIT — do whatever you want with it.