# ros2_perception_mcp

`ros2_perception_mcp` is a dedicated, read-only-first MCP server for bounded,
semantic inspection of ROS 2 perception systems.

Version 0.1.0 targets:

- Ubuntu 24.04
- Python 3.12
- ROS 2 Jazzy
- MCP Python SDK 2.x
- stdio transport

The project is intentionally designed as a dedicated perception MCP server
rather than a generic ROS 2 interface.

## Current Status

The current v0.1.0 development status is:

```text
Phase 1  - Project foundation                         COMPLETE
Phase 2  - Architecture and scope                     COMPLETE
Phase 3A - Domain models                              COMPLETE
Phase 3B - Application ports and service boundary     NEXT
Phase 3  - Domain models and application ports        IN PROGRESS
```

Phase 3A implements and verifies the vendor-neutral perception domain model.

Focused Phase 3A verification:

```text
12 passed
```

The project intentionally exposes no perception MCP tools, resources, prompts,
ROS subscriptions, or physical sensor integrations yet. These capabilities are
introduced only in their corresponding roadmap phases.

---

## Architecture

The intended architecture is:

```text
MCP Client
    |
    | stdio
    v
MCP Server
    |
    v
Semantic Perception MCP Surface
    |
    v
PerceptionService
    |
    +--------------------+
    |                    |
    v                    v
Domain Models      Safety / Bounds
    ^
    |
Application Ports
    ^
    |
RosPerceptionAdapter
    ^
    |
JazzyRosPerceptionAdapter
    |
    v
ROS 2 Jazzy
    |
    +----------------------+
    |                      |
    v                      v
RealSense D435i       RPLIDAR A2M8
verification          verification
```

Dependencies point inward.

The domain and application layers form the vendor-neutral semantic core.

ROS 2, MCP, and physical sensor integrations remain adapters around that core.

The domain layer must not depend on:

- `rclpy`
- ROS message packages
- `tf2`
- MCP SDK types
- RealSense SDKs
- SLAMTEC SDKs
- OpenCV
- device-specific APIs

The RealSense D435i and RPLIDAR A2M8 are planned physical verification devices,
not public API dependencies.

---

## Scope

`ros2_perception_mcp` owns bounded semantic inspection of ROS 2 perception
systems.

The planned v0.1.0 scope includes:

- sensor discovery
- stream discovery
- semantic sensor metadata
- stream metadata
- camera metadata
- `CameraInfo`-derived calibration metadata
- depth metadata
- `PointCloud2` metadata
- `LaserScan` metadata
- frame relationships
- freshness evidence
- observed rate evidence
- sensor-health evidence
- diagnostics
- explicitly bounded samples or snapshots

The MCP surface is intended to expose semantic perception operations rather
than raw unrestricted ROS interfaces.

---

## Explicit Boundaries

Version 0.1.0 will not expose:

- arbitrary ROS topic access
- arbitrary ROS topic publication
- arbitrary ROS service calls
- arbitrary ROS action calls
- parameter mutation
- process execution
- launch execution
- shell commands
- camera configuration
- LiDAR configuration
- LiDAR motor control
- motor control
- robot movement
- manipulator movement
- unrestricted payload forwarding
- full-rate image streaming
- full-rate point-cloud streaming

The project is read-only-first.

Inspection must not configure devices or cause actuation.

---

## Responsibility Separation

The ROS 2 MCP projects intentionally have separate responsibilities.

```text
ros2_mcp
    -> generic bounded ROS 2 inspection

ros2_control_mcp
    -> ros2_control semantics

ros2_manipulator_mcp
    -> manipulator-specific semantics

ros2_perception_mcp
    -> perception and sensor semantics
```

Generic ROS access belongs in `ros2_mcp`.

Control semantics belong in `ros2_control_mcp`.

Manipulator semantics belong in `ros2_manipulator_mcp`.

Perception-specific semantic inspection belongs in `ros2_perception_mcp`.

This separation prevents the individual MCP servers from becoming
unbounded general-purpose robot interfaces.

---

## Out of Scope for v0.1.0

The following higher-level perception and robotics capabilities are explicitly
outside the v0.1.0 scope:

- object detection
- segmentation
- pose estimation
- SLAM
- Nav2
- MoveIt
- IMU support

These capabilities may be considered separately in future architecture work but
are not part of the current v0.1.0 contract.

---

## Phase 3A Domain Foundation

Phase 3A implements the pure Python vendor-neutral perception domain in:

```text
src/ros2_perception_mcp/domain/
```

The primary implementation is:

```text
src/ros2_perception_mcp/domain/models.py
```

The domain currently contains:

- `SensorDescriptor`
- `StreamDescriptor`
- `CameraDescriptor`
- `CameraIntrinsics`
- `DepthDescriptor`
- `PointCloudDescriptor`
- `PointCloudField`
- `LaserScanDescriptor`
- `FrameDescriptor`
- `FreshnessStatus`
- `SensorHealth`

Two finite application-owned semantic states are represented using Python 3.12
`StrEnum`:

- `FreshnessCategory`
- `HealthCategory`

Open-ended classifications such as sensor kinds, stream kinds, encodings,
message categories, point-cloud datatypes, and frame identifiers deliberately
remain extensible string values.

---

## Domain Design Principles

Phase 3A follows several important design rules.

### Vendor-neutral

Domain behavior does not depend on a RealSense D435i, RPLIDAR A2M8, or any
other specific device.

### ROS-independent

ROS messages and `rclpy` objects do not appear in the domain API.

ROS 2 Jazzy adapters will later convert ROS observations into semantic domain
objects.

### MCP-independent

Domain models do not contain MCP SDK or protocol types.

MCP is an external adapter around the application and domain layers.

### Immutable

Domain models use frozen dataclasses.

Collections that belong to immutable domain values use tuples.

### Incomplete metadata is representable

Unknown metadata is represented explicitly rather than fabricated.

For example, camera resolution, depth ranges, calibration information, and
frame relationships may be `None` where appropriate.

### Structural validation only

The domain validates deterministic structural invariants.

It does not invent:

- hardware limits
- vendor limits
- freshness thresholds
- rate thresholds
- physical-safety rules

---

## Freshness and Health

Freshness and health are evidence-oriented.

`FreshnessStatus` represents:

- observation time
- age
- evidence
- optional semantic category

Freshness thresholds are not embedded in the domain model.

Threshold configuration and category derivation belong to later application and
safety/bounds work.

`SensorHealth` represents:

- availability
- freshness evidence
- rate evidence
- findings
- semantic health category

A health result is not a physical-safety certification.

The server must never interpret sensor health as authorization for robot
movement or other actuation.

---

## Bounded Data

Perception systems can produce large continuous data streams.

`ros2_perception_mcp` is not intended to forward those streams unrestricted to
an MCP client.

The intended architecture is:

```text
Continuous ROS 2 perception stream
                |
                v
         ROS adapter observes
                |
                v
     Semantic metadata or
        bounded sample
                |
                v
          MCP response
```

Image, depth, point-cloud, and laser-scan access must remain explicitly bounded.

Full-rate streaming is outside the v0.1.0 scope.

---

## Planned Hardware Verification

Two physical sensors are planned for later v0.1.0 verification.

### RealSense D435i

Planned for:

```text
Phase 14 - Real-hardware verification — RealSense D435i
```

Expected verification areas include camera, depth, calibration, stream
metadata, frames, freshness, and bounded perception inspection.

### RPLIDAR A2M8

Planned for:

```text
Phase 15 - Real-hardware verification — RPLIDAR A2M8
```

Expected verification areas include laser-scan metadata, frames, freshness,
rate evidence, health evidence, and bounded scan inspection.

These devices verify the vendor-neutral architecture.

They do not define it.

---

## Run the Foundation Server

Install/synchronize the project environment:

```bash
uv sync
```

Run the current foundation server:

```bash
uv run ros2-perception-mcp
```

The process waits for MCP JSON-RPC on standard input.

At the current development stage, it intentionally advertises no perception
MCP capabilities.

Set:

```text
ROS2_PERCEPTION_MCP_CONFIG
```

to select an alternate TOML configuration file.

---

## Development Tests

`pytest` is maintained as a development dependency.

The focused Phase 3A domain tests can be run with:

```bash
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
uv run python -m pytest -q tests/test_domain_models.py
```

Verified Phase 3A result:

```text
............                                                             [100%]
12 passed in 0.01s
```

Automatic third-party pytest plugin loading is disabled for this focused domain
test because a ROS 2 Jazzy environment can expose unrelated ROS testing plugins
such as `launch_testing`.

ROS-specific testing will be introduced explicitly in the corresponding later
phases.

---

## Project Roadmap

The v0.1.0 roadmap is:

1. Project foundation — COMPLETE
2. Architecture and scope — COMPLETE
3. Domain models and application ports — IN PROGRESS
   - Phase 3A - Domain models — COMPLETE
   - Phase 3B - Application ports and service boundary — NEXT
4. ROS 2 Jazzy adapter foundation
5. Sensor discovery and inspection
6. Camera / Image / CameraInfo
7. Depth
8. PointCloud2
9. LaserScan
10. TF / Frames / Freshness / Rate / Health
11. MCP Tools / Resources / Prompts
12. Safety bounds and diagnostics
13. Focused software verification
14. Real-hardware verification — RealSense D435i
15. Real-hardware verification — RPLIDAR A2M8
16. Final audit, documentation and v0.1.0 release readiness

Each phase requires explicit scope and must preserve the read-only, bounded,
vendor-neutral architecture.

---

## Documentation

Detailed development records are maintained in:

- [Phase roadmap](docs/README_PHASES.md)
- [Phase 1 - Project Foundation](docs/README_PHASE_1.md)
- [Phase 2 - Architecture and Scope](docs/README_PHASE_2.md)
- [Phase 3 - Domain Models and Application Ports](docs/README_PHASE_3.md)

The phase documents are intended to record not only implementation progress but
also architectural decisions, explicit exclusions, validation results, and
responsibility boundaries.

---

## Version Assumptions

The project currently targets:

```text
Ubuntu             24.04
Python             3.12
ROS 2              Jazzy
MCP Python SDK     2.x
MCP transport      stdio
```

ROS Python packages remain system dependencies and are deliberately separated
from the vendor-neutral domain layer.

ROS message semantics will be verified against installed and official ROS 2
Jazzy definitions during the ROS adapter and sensor-specific implementation
phases.

RealSense and SLAMTEC driver versions and conventions remain deferred until
their corresponding integration and hardware-verification phases.

---

## Next Step

The next development step is:

```text
Phase 3B - Application ports and service boundary
```

Phase 3B will define the minimal semantic application contracts required by
later ROS 2 Jazzy adapters.

It must preserve the dependency direction:

```text
MCP Adapter
     |
     v
Application Layer
     |
     v
Domain Layer
     ^
     |
ROS 2 Adapter
```

Phase 3B must not introduce ROS subscriptions, hardware access, MCP perception
tools, device configuration, or actuation.