# INSTAR MQTT Camera MCP Server

Async MQTT runtime for INSTAR IP cameras with complete reconnect state machine, connection health separation, runtime hardening, and MCP integration.

Built on `paho-mqtt>=2.1.0` with Paho sync client + isolated CommandExecutor worker. FastMCP-compatible `fastmcp>=0.1.0` async API with Hermes Agent integration support.

## Hermes MCP Integration

The runtime includes a production-ready MCP server consumable by Hermes Agent and other aagent harnesses:

```bash
user  ❯ Using the INSTAR camera integration, read if the red and yellow alarm area are enabled for camera 224. Use the available camera MCP tools. If the areas are disabled please toggle them on.

hermes-agent  ❯ Checked camera 224.

Red area alarm_areas_red_enable was disabled (0, known_but_stale). Yellow area alarm_areas_yellow_enable was disabled (0, known_but_stale).

Toggled them on:

alarm_areas_red_enable set to 1 → first attempt was clamped (camera stayed 0), second read shows it now observed as 1.
alarm_areas_yellow_enable set to 1 → confirmed observed as 1.
Both red and yellow alarm areas are now enabled on camera 224.
```

### Prerequisites

- Hermes Agent v0.21.0+ installed
- INSTAR camera reachable via MQTT broker
- `camera-mcp` installed locally

Install the package to make `camera-mcp` available:

```bash
git clone https://github.com/mpolinowski/instar-mqtt-mcp.git
cd instar-mqtt-mcp
pip install -e .
    Successfully built mqtt-runtime
    Installing collected packages: mqtt-runtime
    Attempting uninstall: mqtt-runtime
        Found existing installation: mqtt-runtime 0.1.0
        Uninstalling mqtt-runtime-0.1.0:
        Successfully uninstalled mqtt-runtime-0.1.0
    Successfully installed mqtt-runtime-0.1.0
```

```bash
which camera-mcp
~/.local/bin/camera-mcp
```

> This installs the `camera-mcp` console script used by Hermes via stdio.

### Installation hierarchy

**Recommended — Catalog install (production)**

Once the server is published to the Hermes catalog:

```bash
hermes mcp install instar-camera
```

The catalog manifest handles git checkout, pinned ref, venv bootstrap, and prompts for MQTT credentials. This is the supported production path.

**Local development / testing**

Use `hermes mcp add` when developing locally or testing without a published catalog entry:

```bash
echo "INSTAR_MQTT_HOST=<your-broker-host>" >> ~/.hermes/.env
echo "INSTAR_MQTT_PORT=<your-broker-port>" >> ~/.hermes/.env
echo "INSTAR_MQTT_USERNAME=<your-mqtt-user>" >> ~/.hermes/.env
echo "INSTAR_MQTT_PASSWORD=<your-mqtt-password>" >> ~/.hermes/.env
echo "INSTAR_MQTT_CLIENT_PREFIX=<your-topic-prefix>" >> ~/.hermes/.env
echo "INSTAR_MQTT_CAMERA_ID=<your-camera-id>" >> ~/.hermes/.env
source ~/.hermes/.env
```

> __NOTE:__ That these env variables will be available to your Hermes Agent. __If__ you try to run `test_application.py` manually the code will fall back to `.env` - make sure you added your MQTT credentials there as well.

```bash
❯ hermes mcp add instar_camera \
  --command "$(which camera-mcp)"

  Connecting to 'instar_camera'...
  ✓ Connected! Found 6 tool(s) from 'instar_camera':
    set_camera_value                Set a camera capability value and wait for camera observatio...
    get_camera_state                Get current observed state of a camera capability.
Returns ...
    wait_camera_state               Wait for camera capability to reach expected value.
Blocks ...
    get_camera_health               Check operational health and connectivity of camera integrat...
    get_camera_image                Retrieve the most recently received image from camera.
This...
    list_camera_capabilities        List all capabilities/settings the camera runtime can operat...
  Enable all 6 tools? [Y/n/select]: y
  ✓ Saved 'instar_camera' to ~/.hermes/config.yaml (6/6 tools enabled)
  Start a new session to use these tools.
```

Hermes will probe the server, list the six tools, and prompt to enable them. After saving, verify:

```bash
hermes mcp list
hermes mcp test instar_camera

 MCP Servers:

 Name             Transport                      Tools        Status
 ──────────────── ────────────────────────────── ──────────── ──────────
 instar_camera    ~/.local/bin/...   all          ✓ enabled

 Testing 'instar_camera'...
 Transport: stdio → ~/.local/bin/camera-mcp
 Auth: none
 ✓ Connected (1360ms)
 ✓ Tools discovered: 6
   set_camera_value            Set a camera capability value and wait for camera obser...
   get_camera_state            Get current observed state of a camera capability.
Returns...
   wait_camera_state           Wait for camera capability to reach expected value.
Blocks...
   get_camera_health           Check operational health and connectivity of camera int...
   get_camera_image            Retrieve the most recently received image from camera.
This...
   list_camera_capabilities    List all capabilities/settings the camera runtime can o...
```

Expected output shows 6 tools discovered and status `✓ enabled`.

Environment variables required by the server:

- `INSTAR_MQTT_HOST` — MQTT broker hostname
- `INSTAR_MQTT_PORT` — MQTT broker port, default 1883
- `INSTAR_MQTT_USERNAME` — MQTT username
- `INSTAR_MQTT_PASSWORD` — MQTT password
- `INSTAR_MQTT_CLIENT_PREFIX` — MQTT topic prefix, default `cameras`
- `INSTAR_MQTT_CAMERA_ID` — Camera ID, default `224`

You can also persist these in `~/.hermes/.env` and reference them via `--env INSTAR_MQTT_HOST=...` on add, or edit `~/.hermes/config.yaml` under `mcp_servers.instar_camera.env`.

### Skill Integration

A Hermes skill `instar-camera` ships with the repo at `skills/instar-camera/SKILL.md`. The skill is independent of MCP installation and teaches the agent when and how to use the camera tools without constructing MQTT topics manually.

For local use, copy the skill to Hermes' skill path:

```bash
mkdir -p ~/.hermes/skills/home-automation/instar-camera
cp ~/Projects/instar-mqtt-mcp/skills/instar-camera/SKILL.md ~/.hermes/skills/home-automation/instar-camera/
```

Hermes will discover it as a local skill and guide the agent to select the correct MCP tools __after you restarted the service__. Once the catalog entry is upstream, the skill remains a separate user-facing asset.

The catalog manifest `optional-mcps/instar-camera/manifest.yaml` defines:
- git install with pinned ref
- bootstrap venv + pip install
- stdio transport `command: "${INSTALL_DIR}/.venv/bin/camera-mcp"`
- auth.env prompts for the six MQTT variables
- six tools default-enabled

See `docs/hermes.md` for complete configuration.

### Available MCP Tools

The server exposes six tools:

- `set_camera_value` - Change camera setting with confirmation
- `get_camera_state` - Get observed state with confidence
- `wait_camera_state` - Wait for state change
- `get_camera_health` - Check broker/camera connectivity
- `get_camera_image` - Retrieve latest periodic snapshot
- `list_camera_capabilities` - Discover camera settings

Hermes registers these as `mcp_instar_camera_*`.

### Skill Integration

A Hermes skill `instar-camera` ships with the repo at `skills/instar-camera/SKILL.md`. For local use, copy it to Hermes skill path:

```bash
mkdir -p ~/.hermes/skills/home-automation/instar-camera
cp skills/instar-camera/SKILL.md ~/.hermes/skills/home-automation/instar-camera/
```

Hermes will discover it as a local skill and guide the agent to use the MCP tools safely without manual MQTT topic construction.

See `skills/instar-camera/SKILL.md` for agent usage guidelines.

## Architecture

- **Paho sync client** with `loop_start()` background network thread
- **Single CommandExecutor worker** owns all Paho calls and state mutations
- **Async callers** enqueue operations and await futures via `loop.call_soon_threadsafe`
- **Ownership invariant**: Executor is the only component allowed to call Paho APIs or mutate application state. Paho callbacks only enqueue inbound events.

## Project Structure

```
mqtt_runtime/
├── camera.py              # Public async API
├── executor.py            # CommandExecutor worker thread
├── paho_bridge.py         # Paho callbacks only enqueue
├── models.py              # StateEntry, Command, Confidence, Events
├── state_cache.py         # Confidence model, disconnect downgrade
├── subscription_registry.py # No wildcards, explicit topics only
├── command_manager.py     # FIFO, idempotency, superseding
├── topic_registry.py      # Topic metadata from topics.py + API docs
├── topics.py              # 740 topic definitions
├── event_queue.py         # Inbound/operation queues
└── experiments.py         # Paho behavior experiments

tests/
├── run_tests.py
└── unit/                  # 14 test suites
    ├── test_ownership.py
    ├── test_state_cache.py
    ├── test_command_lifecycle.py
    ├── test_async_boundary.py
    ├── test_connection_state_machine.py
    ├── test_subscription_registry.py
    ├── test_shutdown.py
    ├── test_concurrency.py
    ├── test_event_stream.py
    ├── test_payload_parsing.py
    ├── test_phase2_reconnect.py
    ├── test_phase2_health.py
    ├── test_phase2_events.py
    └── test_phase2_images.py

main.py                    # Live demo: toggle red alarm area
```

## Quick Start

### Install
```bash
pip install -r requirements.txt
pip install python-dotenv
```

### Broker Configuration

The MQTT broker host, port, username, password, client prefix and camera ID must be provided via environment variables. Do not use hardcoded defaults.

**Production / Hermes**: Configure them in `~/.hermes/.env` via the catalog manifest prompts. They are injected into the MCP server at runtime.

**Local development**: Create a `.env` file in the repository root with the six variables:

```
INSTAR_MQTT_HOST=...
INSTAR_MQTT_PORT=...
INSTAR_MQTT_USERNAME=...
INSTAR_MQTT_PASSWORD=...
INSTAR_MQTT_CLIENT_PREFIX=...
INSTAR_MQTT_CAMERA_ID=...
```

The application automatically loads this file via `python-dotenv`. Do not commit `.env` to version control.

### Live Demo
```bash
python3 main.py
```
Toggles `alarm/areas/red/enable` via plain MQTT and TLS (insecure), verifies state with confidence tracking.

### Programmatic Usage
```python
from mqtt_runtime.camera import Camera

camera = Camera(client_prefix=os.getenv("INSTAR_MQTT_CLIENT_PREFIX"), camera_id=os.getenv("INSTAR_MQTT_CAMERA_ID"))

await camera.connect(
    host=os.getenv("INSTAR_MQTT_HOST"),
    port=int(os.getenv("INSTAR_MQTT_PORT", "1883")),
    username=os.getenv("INSTAR_MQTT_USERNAME"),
    password=os.getenv("INSTAR_MQTT_PASSWORD"),
    use_tls=False,
)

# Relative path resolves via TopicRegistry
state = await camera.get_state("alarm/areas/red/enable")
print(state)  # {'value': '1', 'confidence': 'observed', ...}

await camera.set_value("alarm/areas/red/enable", "0")
await camera.subscribe("alarm/areas/red/enable")

await camera.close()
```

## Public API

- `await camera.connect(host, port, username, password, client_prefix, camera_id, use_tls, tls_insecure)`
- `await camera.close()`
- `await camera.set_value(topic, value, timeout=...)` — topic resolves via registry
- `await camera.get_state(topic)`
- `await camera.wait_for_state(topic, expected, timeout=...)`
- `await camera.subscribe(topic)`
- `await camera.unsubscribe(topic)`
- `await camera.check_health()` — returns HealthResult with broker_connected and camera_online
- `await camera.get_image(topic)` — decode image outside callback path
- `camera.events()` — async iterator for CameraEvent (STATE_CHANGED, CONNECTION_CHANGED, COMMAND_COMPLETED, ERROR)

## Topic Metadata

Catalogue of 663 stateful topics from:
- `mqtt_runtime/topics.py` — authoritative topic list + type/flags
- `topics/api/_extracted.json` — descriptions + value hints

Topic naming:
- Read (status): `{prefix}/{id}/status/<path>` — retained
- Write (command): `{prefix}/{id}/<path>` — publishes `{"val": "..."}`

## Phase 2 Features

### Phase 0 (Complete)
- ✅ Executor ownership model
- ✅ Async bridge with `loop.call_soon_threadsafe`
- ✅ Worker supervision
- ✅ Connection state machine
- ✅ Subscription registry (no wildcards)
- ✅ State confidence model (UNKNOWN / KNOWN_BUT_STALE / OBSERVED)
- ✅ Disconnect downgrade: OBSERVED → KNOWN_BUT_STALE
- ✅ Retained messages never confirm commands
- ✅ Command lifecycle with FIFO per topic
- ✅ Idempotency and superseding
- ✅ Event stream
- ✅ Shutdown resolves all futures

### Phase 1 (Complete)
- ✅ Command confirmation requires OBSERVED confidence tier and non-retained message
- ✅ Same-topic commands produce last-wins final camera state
- ✅ UNCHANGED decided before publish from fresh OBSERVED state
- ✅ CLAMPED as valid observed different value
- ✅ Worker failures resolve pending futures
- ✅ Queued commands may resume after reconnect

### Phase 2 (Complete)
- ✅ Complete reconnect state machine with exponential backoff
- ✅ Subscription restoration hardening with pending additions/removals
- ✅ Connection health separation: broker_connected vs camera_online
- ✅ StateCache production hardening with immutable snapshots and bounded LRU
- ✅ Public event stream API with bounded queue and diagnostics
- ✅ Public async API stabilization
- ✅ Image/content support with LRU cache and base64 validation
- ✅ MCP adapter boundary with Result model

## Testing

```bash
python3 tests/run_tests.py
```

14 test suites, all passing. Tests verify ownership boundaries, async isolation, state transitions, command lifecycle, reconnect behavior, health separation, event ordering, image handling.

## Phase 2 Verification Report

- **Test Suites**: 14 / 14 passing
- **Phase 0 Guarantees**: Preserved
- **Phase 1 Guarantees**: Preserved
- **Phase 2 Guarantees**: Reconnect deterministic, state confidence enforced, MCP adapter ready
- **Remaining**: Empirical Paho runtime verification against live broker

## Migration from mcp_server.py

`mcp_server.py` deleted. Functionality migrated to:
- `mqtt_runtime/topic_registry.py` — TopicRegistry + CameraTopic
- `mqtt_runtime/camera.py` — Async API
- `mqtt_runtime/topics.py` — Moved from `topics/topics.py`

Old synchronous API replaced with async ownership model.

## Security

- Credentials via parameters, never source
- TLS opt-in, cert verification default
- No wildcards allowed in subscriptions
- Worker failure isolates per-operation errors