# nicegui-llm-bridge

> Drive **any** NiceGUI 3.x app from an LLM. No browser. No DOM. Pure wire.

> [!CAUTION]
> **v0.0.1-alpha — YOLO ship, single overnight build, only smoke-tested against the bundled `examples/gallery.py` and one probe against live `nicegui.io`.** No unit tests beyond the round-trip E2E. Wire protocol is NiceGUI-3.x-specific and will break on major-version drift. Feedback from agent operators welcome — see [issue #1](../../issues/1) for the questions we'd most like answered.

This is the third member of the **nicegui-tui / nicegui-wire / nicegui-llm-bridge** trio. It treats NiceGUI's `Client` abstraction as exactly what it is — a consumer of an outbox — and adds an LLM-shaped rendering target:

- `nicegui-tui`  → renders into a Textual TUI **in-process**
- `nicegui-wire` → renders into Textual / framebuffer **over the network**
- `nicegui-llm-bridge` → renders into an **LLM's cognition**, via MCP

## What it does

Connects to a running NiceGUI page via the Socket.IO wire (reusing `nicegui-wire`'s engine), keeps a live element-tree mirror, and exposes it to an LLM as indented bullets:

```
- div#3
  - div#4 text='nicegui-llm-bridge gallery'
  - q-tabs#6 model-value='basics' [interactive: update:modelValue]
    - q-tab#7 label='Basics', name='basics'
  - q-tab-panels#12 model-value='basics' [interactive: update:modelValue]
    - q-tab-panel#13 name='basics'
      - div#17 text='count: 0'
      - q-btn#18 label='Increment' [interactive: click]
      - q-btn#19 label='Warn' [interactive: click]
```

Then the LLM can act: `click(18)` → page rerenders → LLM sees `count: 1`.

## MCP tools

| Tool | Purpose |
|------|---------|
| `open(url)` | Connect to a NiceGUI page and return the indented-bullet view |
| `read()` | Re-emit current mirror (no network roundtrip) |
| `actions()` | JSON list of every interactive element |
| `click(element_id)` | Fire a click; return post-action view |
| `fill(element_id, value)` | Set a form value; return post-action view |
| `choose(element_id, option)` | Pick a radio/select option by visible label |

All element references are **raw integer IDs** (`#18`), surfaced inline so the LLM can never guess — it must read before it acts.

## Why indented bullets

For tree-shaped data, indentation column = depth. No pointer chasing, no global indices to cross-reference, matches the training distribution (READMEs, file trees, doc outlines). Edge lists, JSON, and adjacency lists are all worse for LLM tree reading — see arXiv 2511.10234 (*Lost in Serialization*) for the empirics.

## Snapshot-on-demand

The background wire connection ingests every server update into a local mirror. The LLM never sees the firehose — it only sees what `read()` / `click()` / `fill()` return. A `ui.log` tailing at 10 Hz is invisible until the LLM asks.

## Try it

```bash
# 1. Set up venv + install (one-time)
python3 -m venv .venv
.venv/bin/pip install -e .

# 2. Launch the gallery (terminal A)
.venv/bin/python examples/gallery.py

# 3. Drive it (terminal B)
.venv/bin/python examples/smoke.py     # CLI demo, prints transcript
.venv/bin/python tests/test_mcp_e2e.py # full MCP round-trip
```

## Use from Claude Code

Add to `~/.claude.json` `mcpServers` (or use the bundled `.mcp.json` if you launch Claude Code from this repo):

```jsonc
{
  "mcpServers": {
    "nicegui-bridge": {
      "command": "/Users/evnchn/nicegui-llm-bridge/.venv/bin/python",
      "args": ["-m", "nicegui_llm_bridge.mcp_server"]
    }
  }
}
```

Then in Claude Code: open a NiceGUI app (`examples/gallery.py` in another terminal), and ask Claude to drive it. Claude will call `open` → `actions` → `click`/`fill` like a screen reader would.

## Architecture

```
NiceGUI server ──Socket.IO──> nicegui-wire (tree decode, event encode)
                                        │
                                        ▼
                              ElementTree mirror
                                        │
                                        ▼
                           serialize.render() → bullets
                                        │
                                        ▼
                              Session (snapshot-on-demand)
                                        │
                                        ▼
                              FastMCP tools (stdio)
                                        │
                                        ▼
                                  LLM consumer
```

## Status

v0.0.1-alpha, YOLO-shipped 2026-06-03 ~10:30 HKT. Works against NiceGUI 3.12.x. Probed against `https://nicegui.io` (380 elements in 1.22 s, 15 KB YAML view, 42 actionable elements surfaced).

Known limitations:

- `ui.html` and arbitrary Vue components → render as opaque entries (`nicegui-html#39` with no inner content)
- `ui.aggrid` / `ui.echart` → schema only, not data
- Page navigation (multi-route apps) → not yet; `open()` reconnects from scratch
- Quasar client-side toggles (q-drawer / q-menu burgers) don't roundtrip the wire — they live in DOM, not the outbox
- `q-btn` content nested in `nicegui-html` looks empty to the LLM until we surface `innerHTML` (planned)
- Wire protocol is NiceGUI 3.x specific; major-version drift will break

## Credits

Wire decoding lifted from [nicegui-wire](https://github.com/evnchn/nicegui-wire). Gallery example adapted from [nicegui-tui](https://github.com/evnchn/nicegui-tui).