Webots MCP

Full-access MCP server for the Webots robot simulator. Lets an AI assistant see, understand, and modify a running simulation: scene-tree inspection/editing, node spawning/deletion, viewport screenshots, simulation control, generic control of any robot (motors, sensors, cameras, LEDs, motions), and arbitrary code execution inside Webots.

Architecture

Claude (MCP client)
   │ stdio (MCP protocol)
Python MCP server (server/main.py, FastMCP)
   │ TCP JSON frames, localhost:10022
mcp_bridge — Supervisor controller inside Webots (full scene/simulation API)
   │ localhost:10023
mcp_robot  — generic agent controller, attachable to any robot (auto-discovers devices)

Related MCP server: Desktop Commander MCP Server

Setup

1. pip install -r requirements.txt

2. Register the server with Claude Code (use this repo's actual path):

   claude mcp add webots -- python <path-to-this-repo>/server/main.py

   or add to .mcp.json:

   { "mcpServers": { "webots": { "command": "python", "args": ["<path-to-this-repo>/server/main.py"] } } }

3. Get a world with the bridge running — either:

   • ask for launch_webots (opens the bundled worlds/demo.wbt), or

   • for your own project: install_bridge_into_world("path\\to\\your.wbt"), then open it in Webots.

Whole-software control

Beyond the loaded world, the MCP controls the Webots application itself:

• Any world, any project: launch_webots auto-installs the bridge into worlds that don't have it (backup created). Works on Webots' own sample worlds too (list_sample_worlds).

• Full asset library: search_protos / get_proto_info index all ~850 official PROTOs (robots, furniture, environments, appearances); add_proto_to_world declares one so spawn_node('Nao { }') etc. works after a reload.

• Projects: create_project scaffolds a new Webots project; create_controller writes robot controllers.

• Recording: start/stop_movie_recording (.mp4), start/stop_animation_recording (interactive HTML5), export_screenshot to file.

• Application settings: get_webots_preferences / set_webots_preference (registry-backed: startup mode, python command, rendering options...).

• Process control: launch (windowed/fullscreen/minimized/no-rendering), quit, console capture, docs search.

Dynamic scene observation

The MCP sees the scene in motion, not just as snapshots:

• watch_simulation(duration_s) — run the sim and get a motion digest: every object's trajectory (positions over time), displacement, path length, top speed, plus interaction events (contact_start/contact_end: who touched whom, when, where — attributed by matching contact points across objects).

• start_tracking / get_object_trajectories / get_interactions / stop_tracking — record continuously while you drive robots or apply forces.

• capture_sequence(steps, frames, follow=...) — filmstrip of the 3D view as the sim advances; follow keeps a moving object in frame.

• set_viewpoint(look_at=[x,y,z]) — aim the camera at any point (orientation is computed; Webots cameras look along +x with +z up).

Scene understanding (Unity-MCP-style)

• screenshot_multiview(target=..., batch='surround'|'orbit') — capture 6 canonical views (or an azimuth × elevation orbit grid) around any node or the whole scene in one call, each returned inline with its angle caption; viewpoint restored after.

• get_viewport_screenshot(view_target=..., view_position=[x,y,z]) — positioned one-shot capture: aim at a node (auto-framed from scene bounds) or from an exact position, without permanently moving the camera.

• find_nodes(query, base_type) — search the scene by name/DEF/type substring; returns summaries with world positions.

• get_scene_tree(page_size=..., parent=..., cursor=...) — paged one-level listing for large worlds (returns next_cursor), in addition to the recursive summary.

• get_scene_bounds() — center + radius of the dynamic part of the scene.

• batch_execute([{action, params}, ...]) — run many scene/simulation commands in one call (bulk spawning, mass field edits) with per-command results and stop_on_error control.

• MCP resources — read-only live state at webots://simulation, webots://scene, webots://robots, webots://scene/{node}, and webots://conventions (units + this world's up-axis / coordinate system).

Semantic scene model (scene_model group)

A ready-made semantic picture of the world instead of reconstructing it field by field:

• get_object_catalog() — one inventory row per Solid/Robot: position, yaw, size (from the boundingObject), mass, static|dynamic, color, parent. Paged.

• get_object_properties(node) — deep single-object composite: full world AABB, velocity, center of mass, static balance, contact partners, devices.

• get_contact_points(node) / richer get_node_details (velocity, contact count, CoM) / reset_node_physics(node) — stop a single runaway object.

• Spatial queries: find_nodes_near, objects_in_region, check_overlap, find_overlapping_pairs, get_spatial_relations (touching / on_top_of / inside / near) — answer "what's on the table?" in one call.

World building (world_build group)

Reliable authoring — no more objects spawned floating or intersecting:

• drop_to_ground(node) / place_on(node, target) — rest an object on the support beneath it or centered on another's top face.

• align_objects / distribute_objects / place_row / place_grid — bulk layout.

• find_free_space(size, region) / scatter_objects(count, size, region, seed=...) — collision-aware placement + reproducible domain randomization.

• validate_world() — static lint: overlaps, below-floor / floating objects, dynamic nodes missing a boundingObject, duplicate DEFs, non-ENU warning.

• get_scene_map() — top-down labeled vector SVG of the whole layout.

• snapshot_scene(name) / diff_scene(a, b='now') — "what did my last edit change?" (added / removed / moved / rotated).

Run & understand (analyze group)

Turn "run it and see" into one structured, deterministic call:

• wait_until(condition, timeout_s) — step until an event fires instead of guessing a duration. Condition DSL: distance, contact, speed, position, sim_time, plus any/all combinators. Returns when/where it fired.

• run_experiment(duration_s, watch=, until=, restore=) — one iteration in one call: checkpoint → track → run (for a duration or until a condition) → return a run report (per-object motion + displacement, interaction timeline, scene diff vs start, anomalies) → rewind (auto / keep / on_anomaly). Optional final image.

• detect_anomalies() — over the currently-tracked motion: NaN/inf blow-ups, teleports, runaway velocity, below-floor / out-of-arena — each with a fix hint.

• get_console_diagnostics() / get_controller_logs(robot) — classify the Webots console (ODE/physics, controller tracebacks, missing assets, parse warnings) with fixes, and split logs per controller.

Authoring & automation (authoring group)

Session → reusable assets and reproducible code:

• generate_world_script(format='python'|'json'|'wbt') — export the current scene as a standalone Supervisor script, a declarative scenario, or a clean world file.

• extract_proto_from_node(node, proto_name) — turn a tuned node into a reusable PROTO (exposes translation/rotation/name) written to the project protos/.

• create_world(name, template='empty'|'indoor_room'|'outdoor_flat') — scaffold a ready .wbt (base nodes only); backup_world / list_world_backups / restore_world_backup — managed edit history.

• list_appearances / set_appearance(node, base_color=) / set_recognition_colors — style shapes and make objects visible to camera recognition in one call.

• configure_lighting(preset='indoor'|'outdoor'|'studio'|'night') — fix the top cause of useless screenshots and failed recognition.

• record_states(duration_s, nodes=, fields=) — record motion to a CSV file (with summary stats) for offline analysis and regression tests.

Experiments & scenarios (experiments group)

• configure_physics(recipe=|gravity=|basic_time_step=|random_seed=|fps=) — set WorldInfo without field paths; recipes: earth/moon/mars/zero_g/slow_motion/ high_fidelity. Set random_seed (then reset) for reproducible physics.

• profile_simulation(duration_s) — achieved sim/wall real-time factor; profile_from_log(path) parses a --log-performance file.

• compare_runs(report_a, report_b) — per-object end-position divergence between two run_experiment reports ("it fails one time in five").

• save_scenario(name) / load_scenario(file, seed=) / run_scenario(file, runs=, seeds=) — capture the world as a versionable scenario.json (objects + physics + wait_until conditions + duration), rebuild it deterministically, and batch build→run→report across seeds with a divergence comparison.

Developer experience (dx group)

• get_viewport_labels() — project every object into the CURRENT 3D view and get pixel-space labels (near→far, in-frame); overlay them on get_viewport_screenshot.

• describe_sample(path) / list_sample_worlds(query) — summarize any world file (version, timestep, coordinate system, robots + controllers) and browse the installed Webots sample/benchmark worlds.

• update_world_file(path) / clear_webots_cache() — migrate old worlds and clear the asset cache via the Webots CLI.

• Workflow prompts — inspect_scene, robot_bringup, record_demo, experiment_loop: proven multi-step recipes the client can invoke directly.

• Resource webots://console — the last 50 lines of the Webots console.

Extern controllers & asset import (extern group)

Debug the user's real code and pull in assets without touching the world file:

• run_extern_controller(controller_path, robot) / get_extern_controller_output / stop_extern_controller — set the robot's controller to <extern> and run any controller file as an external process, capturing its stdout: the edit→run→read→iterate controller-development loop.

• create_supervisor_script(name, code) / run_supervisor_script(name, robot) — scaffold and run full-speed Supervisor automation as an extern controller.

• import_cad_model(url, physics=, bounding_box=) — drop an .obj/.dae mesh into the scene as a CadShape Solid.

• convert_proto(proto_file) — flatten an opaque PROTO to base nodes (headless).

High-level robot behaviors (behavior group)

One tool call = one closed-loop behavior (no micro-managing wheel velocities):

• drive_robot(robot, linear, angular, duration_s) — differential-drive convenience; auto-pairs wheel motors, converts (v, ω) → wheel speeds, runs and stops, returns start/end pose + distance.

• move_robot_to(robot, target, tolerance) — closed-loop go-to-point via a heading controller over ground-truth pose (straight-line reactive, not a planner).

• build_occupancy_grid(robot, resolution, size) — rasterize a lidar scan + pose into an ASCII occupancy map + world-frame obstacle list.

Ground-truth perception

Webots cameras can report what they see — no ML needed:

• get_camera_recognition(robot) — labeled detections from a robot camera: model name, node id, 3D pose relative to the camera, physical size, pixel bounding box, colors. Run enable_camera_recognition(robot) once first (adds the Recognition node via the supervisor). Objects need a non-empty recognitionColors field.

• get_segmentation_image(robot) — per-object color mask image (enable_camera_recognition(robot, segmentation=True) first).

• get_depth_image(robot) — RangeFinder depth map as an image + distance stats.

• get_radar_targets(robot) — radar detections (distance, azimuth, speed, power).

Scene authoring extras

• get_node_string(node) — export any node's full "source" (all field values); clone_node(node, new_def, position) — duplicate it in one call.

• get_node_pose(node, relative_to=...) — pose in another node's frame (e.g. cup relative to gripper); include_center_of_mass=True adds CoM + static balance.

• get_selected_node() — the node the user clicked in the Webots GUI: lets a human point at an object for the assistant.

• world_reload() / get_recording_status() — reload world; poll movie encoding.

• save_checkpoint(name) / restore_checkpoint(name) — snapshot all dynamic objects and rewind: try an action, undo, try again.

• set_joint_position(joint, pos) — pose articulated joints through the supervisor, no motors/controllers needed (e.g. pose an arm for a screenshot).

• insert_field_item / remove_field_item — full MF-field editing (append points to coordinate arrays, remove children by index...).

• frame_node(node) — Webots' built-in "move viewpoint to object" fast path.

Robot control extras

• configure_motor — acceleration limits, available force/torque, PID gains [kp, ki, kd], or direct force/torque actuation (bypass position control).

• get_motor_state(..., include_feedback=True) — measured force/torque feedback.

• get_lidar_summary — now includes a polar occupancy digest (nearest obstacle per sector, degrees, null = clear) and optional 3D point cloud.

• export_urdf(robot) — the robot's kinematic model as URDF.

• send_message / get_messages — inter-robot radio (Emitter/Receiver), with signal strength and direction to sender.

• set_connector(lock=...) / vacuum_gripper(on=...) — docking and suction grasping.

• speak(text) — robot text-to-speech; set_brake(damping); display_draw(commands) — draw text/shapes on robot Display devices; get_battery; robot_custom_data.

• set_node_visibility — hide/show objects per viewer (declutter screenshots).

• get_node_proto — introspect a PROTO instance's parameters and derivation chain.

Diagnostics, tool groups & tests

• preflight() — one call that health-checks the stack (Webots process, bridge port, round-trip latency, sim state, agents) and tells you the fix for anything failing. Run it first when something misbehaves.

• list_tool_groups() / manage_tool_groups(group, enabled) — disable tool groups you don't need (observe, app, assets) to keep the tool list lean; core stays on.

• Tests: pip install pytest, then cd server && python -m pytest tests -q (fake-bridge unit tests for every tool module + TCP frame-protocol tests; no Webots required).

Typical workflow

1. get_simulation_state — confirm the bridge is connected

2. get_scene_tree (paged for big worlds) + get_viewport_screenshot / screenshot_multiview — see and understand the scene; find_nodes to search it

3. Edit: spawn_node, move_node, set_node_field, delete_node, set_viewpoint (batch_execute for bulk edits)

4. Control a robot: list_robots → attach_mcp_controller(robot) → get_robot_devices → set_motor / get_camera_image / get_sensor_values

5. Drive time: set_simulation_mode (pause/realtime/fast), step_simulation, reset_simulation

6. Anything else: execute_supervisor_code / execute_robot_code

Notes & limitations

• The Webots installation is auto-detected (WEBOTS_HOME env var → Windows registry → webots on PATH → standard install locations on Windows/Linux/macOS). Set WEBOTS_HOME explicitly only for unusual install paths.

• Ports are configurable: WEBOTS_MCP_PORT (default 10022) and WEBOTS_MCP_AGENT_PORT (default 10023).

• Pause is implemented by the bridge not stepping (the GUI shows "0.00x"). While paused, scene/simulation tools work and step_simulation is deterministic, but per-robot commands may time out — step or resume first.

• Pausing from the Webots GUI blocks the bridge until resumed; prefer the set_simulation_mode tool.

• Viewport screenshots capture the 3D view as rendered — keep the Webots window reasonably large and unobstructed for useful images.

• attach_mcp_controller replaces the robot's own controller (the previous name is returned; restore it with set_node_field(robot, "controller", old_name) + execute_supervisor_code restartController if needed).

• If Webots was force-killed, stale mcp_bridge.py python processes can hold port 10022 and freeze new sessions — kill leftover python.exe mcp_bridge.py processes.

Layout

• server/ — FastMCP server (main.py), TCP client (connection.py), tool modules (tools/)

• controllers/mcp_bridge/ — supervisor bridge controller (runs inside Webots)

• controllers/mcp_robot/ — generic robot agent controller

• worlds/demo.wbt — demo world with bridge + a simple camera/lidar-less wheeled robot