virtual_robotics
Control virtual robots in Unity and VRChat environments to spawn, move, monitor, test navigation, and synchronize with physical robots for simulation and development.
Instructions
Virtual robot control (Unity/VRChat) using existing MCP servers.
This portmanteau tool provides comprehensive virtual robot operations, leveraging unity3d-mcp, vrchat-mcp, and avatar-mcp for scene control, movement, and environment management.
Args: robot_type: Robot model (e.g., "scout", "go2", "g1"). action: Operation to perform: - "spawn_robot": Spawn robot in Unity/VRChat scene - "move": Control virtual robot movement - "get_status": Get virtual robot state - "get_lidar": Get virtual LiDAR scan (Unity physics raycast) - "set_scale": Scale robot size (for size testing) - "load_environment": Load Marble/Chisel environment - "test_navigation": Test pathfinding - "sync_with_physical": Sync vbot state with physical bot robot_id: Robot identifier (auto-generated if not provided). position: Spawn position (x, y, z). scale: Size multiplier (for size testing). environment: Environment name (Marble-generated). platform: Target platform ("unity" or "vrchat"). **kwargs: Additional action-specific parameters.
Returns: Dictionary containing operation result.
Examples: Spawn Scout in Unity: result = await virtual_robotics( robot_type="scout", action="spawn_robot", platform="unity", position={"x": 0, "y": 0, "z": 0} )
Load Marble environment:
result = await virtual_robotics(
action="load_environment",
environment="stroheckgasse_apartment",
platform="unity"
)
Move virtual robot:
result = await virtual_robotics(
robot_id="vbot_scout_01",
action="move",
linear=0.2,
angular=0.0
)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| robot_type | Yes | ||
| action | Yes | ||
| robot_id | No | ||
| position | No | ||
| scale | No | ||
| environment | No | ||
| environment_path | No | ||
| platform | No | unity |
Input Schema (JSON Schema)
{
"properties": {
"action": {
"enum": [
"spawn_robot",
"move",
"get_status",
"get_lidar",
"set_scale",
"load_environment",
"test_navigation",
"sync_with_physical"
],
"type": "string"
},
"environment": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
},
"environment_path": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
},
"platform": {
"default": "unity",
"enum": [
"unity",
"vrchat"
],
"type": "string"
},
"position": {
"anyOf": [
{
"additionalProperties": {
"type": "number"
},
"type": "object"
},
{
"type": "null"
}
],
"default": null
},
"robot_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
},
"robot_type": {
"type": "string"
},
"scale": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"default": null
}
},
"required": [
"robot_type",
"action"
],
"type": "object"
}
Implementation Reference
- Primary handler function for the 'virtual_robotics' tool. This nested function inside register() is decorated with @self.mcp.tool() and dispatches to specialized helper methods based on the 'action' parameter.@self.mcp.tool() async def virtual_robotics( robot_type: str, action: Literal[ "spawn_robot", "move", "get_status", "get_lidar", "set_scale", "load_environment", "test_navigation", "sync_with_physical", ], robot_id: Optional[str] = None, position: Optional[Dict[str, float]] = None, scale: Optional[float] = None, environment: Optional[str] = None, environment_path: Optional[str] = None, platform: Literal["unity", "vrchat"] = "unity", ) -> Dict[str, Any]: """Virtual robot control (Unity/VRChat) using existing MCP servers. This portmanteau tool provides comprehensive virtual robot operations, leveraging unity3d-mcp, vrchat-mcp, and avatar-mcp for scene control, movement, and environment management. Args: robot_type: Robot model (e.g., "scout", "go2", "g1"). action: Operation to perform: - "spawn_robot": Spawn robot in Unity/VRChat scene - "move": Control virtual robot movement - "get_status": Get virtual robot state - "get_lidar": Get virtual LiDAR scan (Unity physics raycast) - "set_scale": Scale robot size (for size testing) - "load_environment": Load Marble/Chisel environment - "test_navigation": Test pathfinding - "sync_with_physical": Sync vbot state with physical bot robot_id: Robot identifier (auto-generated if not provided). position: Spawn position (x, y, z). scale: Size multiplier (for size testing). environment: Environment name (Marble-generated). platform: Target platform ("unity" or "vrchat"). **kwargs: Additional action-specific parameters. Returns: Dictionary containing operation result. Examples: Spawn Scout in Unity: result = await virtual_robotics( robot_type="scout", action="spawn_robot", platform="unity", position={"x": 0, "y": 0, "z": 0} ) Load Marble environment: result = await virtual_robotics( action="load_environment", environment="stroheckgasse_apartment", platform="unity" ) Move virtual robot: result = await virtual_robotics( robot_id="vbot_scout_01", action="move", linear=0.2, angular=0.0 ) """ if action == "spawn_robot": return await self._spawn_robot(robot_type, robot_id, position, scale, platform) elif action == "load_environment": return await self._load_environment(environment or "", platform, environment_path=environment_path) elif action == "get_status": return await self._get_status(robot_id) elif action == "get_lidar": return await self._get_lidar(robot_id) elif action == "set_scale": return await self._set_scale(robot_id, scale) elif action == "test_navigation": return await self._test_navigation(robot_id, environment) elif action == "sync_with_physical": return await self._sync_with_physical(robot_id) else: return format_error_response( f"Unknown action: {action}", error_type="validation_error", details={"valid_actions": ["spawn_robot", "move", "get_status", "get_lidar", "set_scale", "load_environment", "test_navigation", "sync_with_physical"]}, )
- src/robotics_mcp/server.py:355-370 (registration)Registration block in the main server where self.virtual_robotics.register() is called to attach the tool to the FastMCP server instance.def _register_tools(self): """Register all MCP tools.""" # Note: MCP servers are already mounted in __init__ # Register portmanteau tools (SOTA: max 15 tools) self.robotics_system.register() # Portmanteau: help, status, list_robots self.robot_control.register() # Portmanteau: movement, status, control self.virtual_robotics.register() # Portmanteau: virtual robot operations self.vbot_crud.register() # Portmanteau: CRUD for virtual robots self.robot_model_tools.register() # Portmanteau: create, import, export, convert self.robot_animation.register() # Portmanteau: animation and behavior control self.robot_camera.register() # Portmanteau: camera and visual feed control self.robot_navigation.register() # Portmanteau: path planning and navigation self.spz_converter.register() # Portmanteau: .spz file conversion and Unity plugin management logger.info("All tools registered")
- src/robotics_mcp/server.py:127-127 (registration)Instantiation of the VirtualRoboticsTool class with MCP server, state manager, and mounted servers.self.virtual_robotics = VirtualRoboticsTool(self.mcp, self.state_manager, self.mounted_servers)
- Key helper method for spawning virtual robots, supporting both Unity and VRChat platforms via mounted MCP servers or mock fallback.async def _spawn_robot( self, robot_type: str, robot_id: Optional[str], position: Optional[Dict[str, float]], scale: Optional[float], platform: str, **kwargs: Any, ) -> Dict[str, Any]: """Spawn virtual robot in scene. Args: robot_type: Type of robot. robot_id: Robot identifier. position: Spawn position. scale: Robot scale. platform: Target platform. **kwargs: Additional parameters. Returns: Spawn result. """ if not robot_id: robot_id = f"vbot_{robot_type}_01" position = position or {"x": 0.0, "y": 0.0, "z": 0.0} scale = scale or 1.0 # Register robot in state manager robot = self.state_manager.register_robot(robot_id, robot_type, platform=platform, metadata={ "position": position, "scale": scale, "spawned": True, }) logger.info("Spawning virtual robot", robot_id=robot_id, robot_type=robot_type, platform=platform) try: if platform == "vrchat": # Use VRChat OSC to spawn robot in world # VRChat worlds can have spawnable objects controlled via OSC result = await self._spawn_in_vrchat(robot_id, robot_type, position, scale, **kwargs) elif platform == "unity": # Use Unity tools to spawn robot result = await self._spawn_in_unity(robot_id, robot_type, position, scale, **kwargs) else: return format_error_response( f"Unknown platform: {platform}", error_type="validation_error", details={"valid_platforms": ["unity", "vrchat"]}, robot_id=robot_id, ) robot.connected = True self.state_manager.update_robot_status(robot_id, connected=True) return format_success_response( f"Virtual robot {robot_id} spawned in {platform}", robot_id=robot_id, action="spawn_robot", data={ "platform": platform, "position": position, "scale": scale, **result, }, ) except Exception as e: return handle_tool_error("_spawn_robot", e, robot_id=robot_id, context={"platform": platform, "robot_type": robot_type})
- Input schema defined by function parameters with type hints, including Literal enums for action and platform, and detailed docstring describing usage.async def virtual_robotics( robot_type: str, action: Literal[ "spawn_robot", "move", "get_status", "get_lidar", "set_scale", "load_environment", "test_navigation", "sync_with_physical", ], robot_id: Optional[str] = None, position: Optional[Dict[str, float]] = None, scale: Optional[float] = None, environment: Optional[str] = None, environment_path: Optional[str] = None, platform: Literal["unity", "vrchat"] = "unity", ) -> Dict[str, Any]: