Which integrations are available for this server?

Integrates with Intel RealSense depth cameras to provide RGB-D vision capabilities, including scene capture, depth queries, and 3D object localization. Integrates with ROS2 to interface with the RealSense camera driver, providing tools for camera control, workspace checking, and data recording for robotic applications. Integrates with Ultralytics YOLO for AI-powered object detection, enabling zero-shot detection and 3D localization of objects. Integrates with YOLO object detection models (via Ultralytics) for real-time object detection and 3D localization within the scene.

How do I use rosclaw-vision-mcp?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@rosclaw-vision-mcp Take a snapshot of the current view." That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

rosclaw-vision-mcp

by ros-claw

Overview Schema Related Servers Score Discussions

Python

Local

rosclaw-vision-mcp

ROSClaw MCP Server for Intel RealSense RGB-D Camera via ROS2.

Part of the ROSClaw Embodied Intelligence Operating System.

Overview

This MCP server gives LLM agents eyes - the ability to see and understand their physical environment through an Intel RealSense depth camera. It implements the ROSClaw Semantic-HAL (语义硬件抽象层) pattern: the LLM never touches the raw 30Hz sensor stream, only on-demand semantic results.

RealSense Camera
      │ USB3
      ▼
realsense2_camera (ROS2, 30Hz)
      │ Fast lane → VLA/rosbag2 (data flywheel)  ← LLM never sees this
      │ Slow lane → rosclaw-vision-mcp
      ▼
LLM Agent (Claude, GPT-4o)
      │ capture_scene_snapshot() → Base64 JPEG
      │ get_object_3d_coordinates("cup") → {x:0.45, y:-0.12, z:0.05}
      ▼
rosclaw-ur-ros2-mcp
      │ pick_object(0.45, -0.12, 0.05)

Related MCP server: Webcam MCP

Why NOT stream raw data to the LLM?

A raw RealSense stream is ~27 MB/s. MCP over JSON-RPC would crash immediately.

Instead, the LLM calls tools on-demand and receives only compressed semantic results:

A Base64 JPEG image (~50-150 KB) when it needs to "look"
A JSON {x, y, z} coordinate when it needs to "locate an object"
A true/false when it needs to "check if a workspace is clear"

Features

Snapshot capture: Base64-encoded JPEG for multimodal LLMs (Claude vision, GPT-4V)
3D object localization: YOLO-World detection + depth back-projection → XYZ coordinates
Workspace collision checking: Scan a 3D volume for obstacles before arm motion
Depth queries: Get precise distance at any pixel
Data flywheel: Launch/stop rosbag2 recording for LeRobot VLA training data
Thread-safe: rclpy spin in daemon thread, FastMCP in main event loop

Hardware

Field	Value
Camera	Intel RealSense D415 / D435 / D455
Interface	USB3
Protocol	ROS2 via `realsense2_camera` driver
ROS2 Topics	`/camera/color/image_raw`, `/camera/aligned_depth_to_color/image_raw`
Color	640×480 RGB8, 30Hz
Depth	640×480 Z16 (uint16 mm), 30Hz, aligned to color

Installation

# 1. Clone
git clone https://github.com/ros-claw/rosclaw-vision-mcp.git
cd rosclaw-vision-mcp

# 2. Source ROS2 (required)
source /opt/ros/humble/setup.bash

# 3. Install dependencies
uv venv --python /usr/bin/python3.10
source .venv/bin/activate
uv pip install -e .

# 4. Optional: YOLO-World for automatic object detection
uv pip install -e ".[detection]"

Enhanced Features (New!)

✅ Multi-Camera Support — Connect and control multiple RealSense cameras simultaneously
✅ Auto Topic Discovery — Automatically find and connect to available cameras
✅ YOLO Object Detection — Optional AI-powered object detection with 3D localization
✅ Stereo Vision — Capture synchronized images from dual cameras
✅ SSE Transport Mode — Persistent server for stateful connections (fixes stdio state loss)
✅ Systemd Service — Run as a system service with auto-restart
✅ Configurable Topics — Support custom ROS2 topic namespaces

Quick Start

1. Start RealSense camera(s)

Single camera:

source /opt/ros/humble/setup.bash
ros2 launch realsense2_camera rs_launch.py align_depth.enable:=true

Multiple cameras:

./scripts/launch-multi-camera.sh

2. Start MCP Server

Option A: Quick start script

./scripts/start-server.sh sse 8000

Option B: Systemd service

sudo ./scripts/install-systemd.sh $USER
sudo systemctl start rosclaw-vision@$USER

Option C: Manual

source /opt/ros/humble/setup.bash
python3 src/vision_mcp_enhanced.py --transport sse --port 8000

3. Run Demos

# Run all demos
python3 demos/demo_all.py

# Or test individual features
mcporter call rosclaw-vision.discover_cameras
mcporter call rosclaw-vision.connect_multi_camera
mcporter call rosclaw-vision.detect_objects camera_id=camera confidence=0.5

Installation

Prerequisites

Ubuntu 22.04+
ROS2 Humble or Jazzy
Python 3.10+
Intel RealSense SDK 2.0

Install Dependencies

# Clone repository
git clone https://github.com/ros-claw/rosclaw-vision-mcp.git
cd rosclaw-vision-mcp

# Install Python dependencies
pip install -e .

# Optional: Install YOLO for object detection
pip install ultralytics

# Install system service (optional)
sudo ./scripts/install-systemd.sh $USER

Claude Desktop Configuration

Stdio mode (stateless):

{
  "mcpServers": {
    "rosclaw-vision": {
      "command": "bash",
      "args": [
        "-c",
        "source /opt/ros/humble/setup.bash && python /path/to/rosclaw-vision-mcp/src/vision_mcp_server.py"
      ],
      "transportType": "stdio"
    }
  }
}

SSE mode (stateful, recommended for production):

{
  "mcpServers": {
    "rosclaw-vision": {
      "url": "http://127.0.0.1:8000/sse"
    }
  }
}

Configuration

ROS2 Topic Names

If your RealSense camera publishes to different topic names (e.g., with namespace):

# Via environment variables
export ROSCLAW_VISION_COLOR_TOPIC=/camera/camera/color/image_raw
export ROSCLAW_VISION_DEPTH_TOPIC=/camera/camera/aligned_depth_to_color/image_raw
export ROSCLAW_VISION_INFO_TOPIC=/camera/camera/color/camera_info

python src/vision_mcp_server.py

SSE Server Options

# Via command line
python src/vision_mcp_server.py --transport sse --host 0.0.0.0 --port 8080

# Via environment variables
export MCP_TRANSPORT=sse
export MCP_HOST=0.0.0.0
export MCP_PORT=8080
python src/vision_mcp_server.py

Available Tools (Enhanced Edition)

Core Tools (vision_mcp_server.py)

Tool	Description
`connect_vision`	Connect to camera via ROS2
`disconnect_vision`	Disconnect from camera
`capture_scene_snapshot`	Capture RGB snapshot as Base64 JPEG
`get_depth_at_pixel`	Get depth (meters) at a specific pixel
`get_object_3d_coordinates`	Detect object and get 3D XYZ position
`get_scene_description`	Get camera metadata and topic info
`check_workspace_clear`	Check if a 3D volume is obstacle-free
`start_data_recording`	Start rosbag2 recording for data flywheel
`stop_data_recording`	Stop recording and finalize bag file

Enhanced Tools (vision_mcp_enhanced.py) ⭐ New!

Tool	Description
`discover_cameras`	🔍 Auto-discover all available RealSense cameras
`connect_multi_camera`	🔗 Connect multiple cameras simultaneously
`capture_from_camera`	📷 Capture from specific camera by ID
`get_camera_status`	📊 Get status of all connected cameras
`detect_objects`	🎯 YOLO object detection with 3D localization
`capture_stereo_image`	🎬 Capture synchronized stereo pair
`disconnect_all`	🔌 Disconnect all cameras

Tool Usage Examples

Auto-discover cameras:

mcporter call rosclaw-vision.discover_cameras

Connect all detected cameras:

mcporter call rosclaw-vision.connect_multi_camera

Detect objects with YOLO:

mcporter call rosclaw-vision.detect_objects camera_id=camera confidence=0.5

Capture stereo image:

mcporter call rosclaw-vision.capture_stereo_image \
    left_camera=camera \
    right_camera=camera_2 \
    quality=85

Available Resources

Resource	Description
`vision://status`	Camera status, resolution, intrinsics
`vision://topics`	ROS2 topic list and types
`vision://connection`	Connection status

End-to-End Pick & Place Example

User: "抓起桌上的红色杯子" ("Pick up the red cup on the table")

LLM workflow:
1. capture_scene_snapshot()
   → Base64 JPEG image

2. get_object_3d_coordinates("red cup")
   → {"x": 0.45, "y": -0.12, "z": 0.38, "confidence": 0.92}

3. check_workspace_clear(x_min=0.3, x_max=0.6, ...)
   → "✓ Workspace clear"

4. [switch to rosclaw-ur-ros2-mcp]
   pick_object(0.45, -0.12, 0.38)
   → "✓ Object picked"

Object Detection

get_object_3d_coordinates() uses a two-stage strategy:

With YOLO-World (pip install ultralytics): Zero-shot detection - works for any object name without training. Finds bounding box, samples depth at center, back-projects to 3D.
Without YOLO-World: Returns the captured frame as Base64 for the LLM to visually locate the object, then the user can call get_depth_at_pixel(u, v) to get 3D coordinates.

Data Flywheel

The start_data_recording() tool captures:

/camera/color/image_raw - RGB video
/camera/aligned_depth_to_color/image_raw - Depth video
/joint_states - Robot arm state
/tf - Transforms

This data feeds the LeRobot pipeline for VLA model training (π0, OpenVLA).

Dependencies

Python 3.10+
ROS2 Humble or Jazzy
mcp[fastmcp]>=1.0.0 - MCP framework
Pillow>=10.0 - JPEG encoding (replaces heavy cv_bridge dependency)
numpy>=1.24 - Array operations
ultralytics>=8.0 (optional) - YOLO-World object detection

Architecture

vision_mcp_server.py
├── VisionState      - Frame data (RGB bytes, depth bytes, intrinsics)
├── StateBuffer      - Thread-safe ring buffer (10 frames)
├── VisionROS2Bridge - rclpy.Node
│   ├── _color_callback()  - /camera/color/image_raw subscriber
│   ├── _depth_callback()  - /camera/aligned_depth_to_color subscriber
│   ├── _info_callback()   - /camera/color/camera_info subscriber
│   ├── get_jpeg_base64()  - RGB → JPEG → Base64
│   ├── get_depth_meters() - Z16 depth lookup
│   ├── pixel_to_3d()      - Pinhole back-projection
│   └── check_volume_clear() - Obstacle detection
└── MCP Tools        - FastMCP @mcp.tool() definitions

License

MIT License - See LICENSE

Part of ROSClaw

rosclaw-vision-mcp - RealSense camera (ROS2)
rosclaw-g1-dds-mcp - Unitree G1 (DDS)
rosclaw-ur-ros2-mcp - UR5 arm (ROS2)
rosclaw-gimbal-mcp - GCU Gimbal (Serial)

This server cannot be installed

license - permissive license

quality - not tested

maintenance

How are these scores calculated?

Maintenance

–Maintainers

–Response time

–Release cycle

–Releases (12mo)

Commit activity

Resources

GitHub Repository

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly
Lightport: Open-Sourcing Glama's AI Gateway
By punkpeye on April 27, 2026.
OpenAI
open source

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ros-claw/rosclaw-vision-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server