# idm-mcp

MCP server for an [ISOTEL IDM](https://isotel.eu) smart gateway. It exposes the
gateway's **devices, parameters, records, protocol stacks, streams and
connections** to an MCP client (Claude Desktop, Chatbox, an agent, anything
speaking MCP).

**Read-only by default** - 14 tools that only GET. Writing is opt-in:
`--allow-write` registers the write tools, `IDM_WRITABLE` names the parameters
that may be changed, and every value is validated against the gateway's own
options (read-only flag, allowed values, numeric range) before anything is sent.

Built on [`isotel-idm`](https://pypi.org/project/isotel-idm/) and `fastmcp`;
target gateway API 1.3.845.

## Workflow

1. `idm_list_devices` — what is attached.
2. `idm_get_device_html` for any device you are about to read or change. It is
   the description layer: sections (System, Acquire, Trigger, Channels, ...),
   units, live values and free-text diagnostics, with every value tagged by its
   parameter name. It honours the access level, so it grows from 140 to 195
   parameters with `--enable-devel --enable-hidden`.
3. Then the value tools.

Skip step 2 only when the user has already named a specific parameter and value
— the intent is unambiguous then. Otherwise describe first and let the sections
inform which parameters matter; don't infer meaning from a parameter name alone.
The write tools return `device_described` and a `hint` when it is false.

## Tools

| Tool | Returns |
|---|---|
| `idm_gateway_info()` | host, API version, mode, group, active and known device counts |
| `idm_list_devices(include_inactive=False, brand_filter=None)` | `[{name, device, root, status}]` |
| `idm_device_status(device)` | `state`, `port`, `endpoints`, `time`, `recording`, counters |
| `idm_get_protocols(device, full=False)` | protocol stack layers, plus streams with `tcp_port` / `gateway_host` |
| `idm_get_streams(device=None)` | every TCP stream with its name, port, clients, rx/tx |
| `idm_list_connections()` | every PHY connection with state and link counters, including one with no device |
| `idm_get_device_html(device, max_bytes=8192)` | the gateway's own HTML page for a device — the description step; honours the access level |
| `idm_get_activity(limit=20)` | recent gateway activity log — the receipt for a write |
| `idm_list_parameters(device, prefix=None)` | parameter paths only — for discovery |
| `idm_get_parameters(device, prefix=None, names=None, fresh=False)` | flattened `{"ec/E": {"value", "unit"}}` |
| `idm_get_structure(device, path="", options=False)` | a whole substructure in one request, e.g. `cfg` or `ch/PGA_in` |
| `idm_get_value(device, parameter, fresh=False, options=False)` | one parameter: value, unit, accuracy, timestamp; with `options=True` also `type` / `allowedValues` / `readOnly` |
| `idm_get_history(device, parameter, time_from, time_to, max_points, scale, average, calc, interpolate)` | recorded samples, aggregated by the gateway |

`device` accepts the root (`device0`), the full name (spaces or underscores) or
an unambiguous substring. Ambiguity is refused with the candidate list rather
than silently picking the first device. Parameter paths use slashes (`ec/E`);
dots (`ec.E`) are accepted too.

`idm_list_parameters` reports variables of a device that wraps them in a `data`
container as `data/ch/PGA_in/PGA_in0`, but the gateway addresses that variable as
`ch/PGA_in/PGA_in0`. The tools accept either and strip the container, so paths
from `idm_list_parameters` can be used as-is.

### History modes

The gateway does the aggregation, so a month or a day costs one call:

| Arguments | Mode | Result |
|---|---|---|
| `max_points=N` | regular | N samples (`limit`): first N after `time_from`, or last N before `time_to`; gateway default 100 |
| `scale=N` | scale | N points spanning the span, each with `m` = records in its interval |
| `scale=N, average=3600` | average | N points averaged over 3600 s intervals, with `n` and `sd` |
| `calc=avg` | calc | one aggregate over the whole record set (`n_sum` = inputs) |

`average` is an **interval duration in seconds** — the Python library's docstring
calls it a record count, which is wrong. `max_points` and `scale` are mutually
exclusive; `average` and `interpolate` require `scale`.

### Streams

`idm_get_protocols` flattens the protocol stack of the port a device sits on:
`PHYSICAL -> VIRTUALPHY -> MESSAGE -> CUSTOM`, with each layer's `rx`/`tx`
counters. A layer that carries a `tcp_port` is a **stream** served by the
gateway — connect to `<gateway_host>:<tcp_port>` for the raw byte stream:

```json
"streams": [{"name": "UART 115200-8E1 to Amplifier", "layer": "CUSTOM",
             "tcp_port": 33100, "gateway_host": "gateway", "clients": []}]
```

Only active devices have a connection port, so this needs one. `full=True`
returns the tree exactly as the gateway sends it.

### Write mode

Read-only is the default and the write tools are then **not registered at all**.
Enable them with `--allow-write` (alias `--allow-post`) or `IDM_ALLOW_WRITE=1`:

```bash
python -m idm_mcp.server --transport http --port 8003 --allow-write
```

| Tool | Effect |
|---|---|
| `idm_command(command, parameters)` | sends a terminal command (`POST /api/v1/terminal/commands`) |
| `idm_open_stream(device, stream)` | gateway dials the stream: `openTCPConnection(host, port)` |
| `idm_close_stream(device, stream)` | closes it: `closeConnection("host:port")` |
| `idm_open_serial(target, preset=None)` | opens a serial port or preset (`openSerialPort`) |
| `idm_close_serial(connection_id)` | closes a serial connection by id (`closeConnection`) |
| `idm_set_value(device, parameter, value)` | writes one parameter — **only** if allow-listed (see below) |
| `idm_set_structure(device, values, path="")` | writes several parameters in one POST — the struct `idm_get_structure` returns, minus `status`/`time` |

**Opening a stream makes the gateway connect to it.** `idm_open_stream` sends
`openTCPConnection(host, port)`; the gateway dials its own endpoint and creates a
PHY connection named `<host>:<port>`. If a device stack sits behind that stream,
it loads as a new device — the stacking case: opening TestPort's
`UART 115200-8E1 to Amplifier` brings up a second `IOLite-4CFB-` (same serial)
over that UART, and `idm_close_stream` (`closeConnection("host:port")`) unloads
it again. The host defaults to `localhost` because the gateway dials itself;
override with `IDM_STREAM_HOST`. `idm_read_stream` is the different operation: it
attaches from *this* host to read raw bytes, and detaches afterwards.

**Every write returns its activity entries.** The gateway's response body is
just `{"result": "OK"|"Error"|"null"}`, so a command's real outcome comes from
`GET /me/activity`: the client snapshots the newest log timestamp before the
POST and returns entries logged after it. A `Command not found` or
`Exception while executing` entry makes the stream/connection tools raise
instead of reporting a silent no-op.

Command names carry **no `$` prefix** on the wire — `help`, not `$help`.

Device **parameter** writes are deliberately not implemented: they need an
explicit allowlist of parameters and are not part of this surface yet.

### Access level

The gateway hides whole sections behind three access flags, and the client passes
your setting both when it opens the device and on every read, so the level can
never differ between one call and the next:

```bash
python -m idm_mcp.server --enable-advanced --enable-hidden --enable-devel
```

| Flag | Env | Reveals |
|---|---|---|
| `--enable-advanced` | `IDM_ADVANCED` | advanced sections |
| `--enable-hidden` | `IDM_HIDDEN` | hidden sections — `cfg` (`rsv`, `release`) needs this |
| `--enable-devel` (alias `--enable-development`) | `IDM_DEVELOPMENT` | development sections — a substructure GET like `/ch.json` is refused without it |

All three are **off by default**, which is the gateway's own narrow view. On
`device4` the default shows 142 parameters and `cfg` comes back empty; with all
three it shows 195 and `cfg` returns `{"rsv": "0x28", "release": "2"}`.

### Parameter writes

Two write paths, same allowlist and same validation: `idm_set_value` for one
variable, `idm_set_structure` for a whole section in a single POST.

**Substructures keep their wrapper key.** `get("")` on `ch/PGA_in` answers
`{"status", "PGA_in": {...}, "time"}`. Send that back minus `status` and `time`,
wrapper included. An unwrapped payload is accepted and wrapped automatically,
because the gateway otherwise applies the *first* value to every sibling — all
four channels took the same gain when I tried it. After the POST the tool
re-reads the section and returns `verified` plus `mismatch`, so a write that did
not take is visible rather than assumed.

`idm_set_value` refuses everything unless the parameter is on an allowlist you
define, and the allowlist lives outside the code:

```bash
IDM_WRITABLE="device4:ch/PGA_in/PGA_in0,device4:ch/ADC/ADC0,*:ch/range/*" \
IDM_ALLOW_WRITE=1 pm2 restart idm-mcp --update-env
```

Each entry is `device:parameter` where `device` is a root (`device4`), a full
name, or `*`; `parameter` is a path, optionally ending in `*` for a prefix. An
empty allowlist means nothing is writable.

Validation before the POST uses the gateway's own `options` for that parameter
(read them with `idm_get_value(..., options=True)`):

- `readOnly: true` → refused;
- enum → the value must be in `allowedValues`, matched case-insensitively and
  sent in canonical spelling (`vin` → `Vin`);
- numeric → must parse, and respect min/max when the gateway reports them.

The payload is the gateway's own shape, `{"<variable>": {"value": "..."}}`,
POSTed to the parameter's URL.

### Freshness

`idm_get_value` and `idm_get_parameters` return the gateway's **last stored
sample** unless `fresh=True`, which sends `after=now` and forces a query to the
device (one round-trip over USB/Bluetooth, so slower).

## Install

```bash
pip install -e . --no-deps
```

`--no-deps` because `isotel-idm` and `fastmcp` are already in that venv.

## Run

**stdio** — the client spawns the process:

```bash
~/.venv/bin/idm-mcp                      # or: python -m idm_mcp.server
```

**HTTP** — long-running, e.g. under pm2:

```bash
python -m idm_mcp.server --transport http --host 127.0.0.1 --port 8003 --path /mcp
```

Ends up at `http://127.0.0.1:8003/mcp`.

### Environment

| Variable | Default | Meaning |
|---|---|---|
| `IDM_GATEWAY` | `http://localhost:33000` | gateway base URL |
| `IDM_GROUP` | `me` | group (`me` = local devices) |
| `IDM_TIMEOUT` | `60` | request timeout, seconds |
| `IDM_USERNAME` / `IDM_PASSWORD` | unset | basic auth, if the gateway needs it |
| `IDM_ALLOW_WRITE` | unset (off) | register the write tools (`1`/`true`/`yes`/`on`) |
| `IDM_WRITABLE` | unset (nothing) | allowlist of `device:parameter` entries for the write tools |
| `IDM_ADVANCED` / `IDM_HIDDEN` / `IDM_DEVELOPMENT` | unset (off) | access level; same as `--enable-*` |
| `IDM_STREAM_HOST` | `localhost` | host the *gateway* dials for `idm_open_stream` (it connects to its own endpoint) |
| `IDM_OPEN_SERIAL_CMD` / `IDM_CLOSE_SERIAL_CMD` | `openSerialPort` / `closeConnection` | command names used by `idm_open_serial` / `idm_close_serial`, for firmware that spells them differently |
| `IDM_MCP_TRANSPORT` / `IDM_MCP_HOST` / `IDM_MCP_PORT` / `IDM_MCP_PATH` | `stdio` / `127.0.0.1` / `8003` / `/mcp` | transport defaults |
| `IDM_PYTHON` | `python3` | interpreter used by `ecosystem.config.cjs` (pm2 only) |
| `IDM_LIVE` | unset | `1` runs the live tests against `IDM_GATEWAY` |

### Command line

Every setting can also be given as a flag; a flag wins over its environment
variable.

| Flag | Equivalent |
|---|---|
| `--gateway URL` | `IDM_GATEWAY` |
| `--group NAME` | `IDM_GROUP` |
| `--timeout SEC` | `IDM_TIMEOUT` |
| `--username` / `--password` | `IDM_USERNAME` / `IDM_PASSWORD` |
| `--allow-write` (alias `--allow-post`) | `IDM_ALLOW_WRITE=1` |
| `--enable-advanced` | `IDM_ADVANCED=1` |
| `--enable-hidden` | `IDM_HIDDEN=1` |
| `--enable-devel` (alias `--enable-development`) | `IDM_DEVELOPMENT=1` |
| `--transport stdio\|http` | `IDM_MCP_TRANSPORT` |
| `--host` / `--port` / `--path` | `IDM_MCP_HOST` / `IDM_MCP_PORT` / `IDM_MCP_PATH` |

`IDM_WRITABLE` deliberately has **no** flag: an allowlist belongs in one place,
not on a command line where it can be mistyped or silently lost from the pm2
environment.

CLI flags override the environment.

### Client configuration (stdio)

```json
{
  "mcpServers": {
    "isotel-idm": {
      "command": "/path/to/venv/bin/idm-mcp",
      "env": { "IDM_GATEWAY": "http://localhost:33000", "IDM_GROUP": "me" }
    }
  }
}
```

### pm2

```bash
pm2 start ecosystem.config.cjs
pm2 logs idm-mcp
pm2 restart idm-mcp
pm2 delete idm-mcp
```

```bash
IDM_ALLOW_WRITE=1 IDM_GATEWAY=http://gateway:33000 pm2 restart idm-mcp --update-env
```

`ecosystem.config.cjs` runs the HTTP transport on `127.0.0.1:8003/mcp` with
`autorestart`. It uses `__dirname` for its working directory and takes the
interpreter from `IDM_PYTHON` (default `python3` on PATH), so start it with the
environment that has `isotel-idm` and `fastmcp` installed:

```bash
IDM_PYTHON=/path/to/venv/bin/python pm2 start ecosystem.config.cjs
```

## Test

```bash
~/.venv/bin/python -m pytest              # 111 unit tests, no gateway needed
IDM_LIVE=1 ~/.venv/bin/python -m pytest   # + 6 tests against the real gateway
~/.venv/bin/python scripts/smoke_http.py  # exercises a running HTTP server
```

`tests/test_server.py` fails if any write operation (`set`, `post`, `delete`,
user/group management, scripts) appears in the package source, and if the tool
set changes — read-only is a tested property, not a promise.

## Behaviour worth knowing

- **Only active devices serve live telemetry.** `idm_get_history` also accepts
  inactive devices, because the gateway keeps their records. A gateway lists
  inactive entries for devices it knows but is not running (they have no
  `root`), and those entries mirror active devices, so active always wins in
  resolution.
- **`Device` objects are cached** per name, and one `Group` keeps a keep-alive
  connection; the client is thread-safe. Device construction redirects stdout,
  because `isotel.idm` prints an inactive-device warning that would corrupt the
  JSON-RPC stream on the stdio transport.
- **`isotel-idm` pins the exact API revision** (1.3.845). A gateway firmware
  update that bumps the revision makes the library refuse to connect; upgrading
  `isotel-idm` is then required.
- **Not included:** service/user/group management, script execution, virtual
devices, firmware upgrade and `show`/`aggr` group-link vertical aggregation.

## Requirements

Python 3.10+, `fastmcp`, and `isotel-idm` **1.3** for a gateway running API 1.3.
The library enforces an API major-version match when it opens the group, so
`isotel-idm` 1.2 installs fine but refuses to connect to a 1.3 gateway
(`Major API versions not compatible, gateway: 1.3 library: 1.2`).

PyPI currently publishes up to **1.2**, so the dependency is declared `>=1.2`:
that keeps `pip install -e .` valid while 1.3 is distributed privately — an
existing 1.3 install already satisfies the requirement and pip will not
downgrade it. The pin moves to `>=1.3` once 1.3 is on PyPI.

## License

LGPL-3.0-only, matching `isotel-idm`. See [LICENSE](LICENSE).