# UI Bridge MCP

MCP (Model Context Protocol) server for UI Bridge - enables AI to inspect and interact with UI elements.

## Overview

This MCP server provides tools for:

- **Inspecting UI elements** - Get element positions, bounds, visibility, and state
- **Interacting with elements** - Click, type, focus, hover
- **Two modes of operation**:
  - **Control mode**: Interact with the qontinui-runner's own Tauri webview
  - **SDK mode**: Interact with external apps via the UI Bridge SDK

## Installation

```bash
# Using pip
pip install ui-bridge-mcp

# Using poetry
poetry add ui-bridge-mcp
```

## Prerequisites

The MCP server requires the **qontinui-runner** to be running, by default at
`127.0.0.1:9876` (see [Environment Variables](#environment-variables)).

For external app access, the target application must have the UI Bridge SDK integrated.

## Configuration

### Claude Desktop

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "ui-bridge": {
      "command": "ui-bridge-mcp"
    }
  }
}
```

### Claude Code

Add to your MCP settings:

```json
{
  "mcpServers": {
    "ui-bridge": {
      "command": "ui-bridge-mcp"
    }
  }
}
```

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `QONTINUI_RUNNER_HOST` | `127.0.0.1` | Runner hostname. The runner listens on the IPv4 loopback only; see [Reaching a Windows runner from WSL](#reaching-a-windows-runner-from-wsl). |
| `QONTINUI_RUNNER_PORT` | `9876` | Runner port. Empty or blank means the default; a value that is not a port (1-65535) is an error naming this variable. |

#### Reaching a Windows runner from WSL

The runner binds `127.0.0.1` only, and its Host gate refuses, among others, a
request whose `Host` is neither a loopback name with the runner's bound port nor a
`QONTINUI_RUNNER_ALLOWED_HOSTS` entry (`403 HOST_NOT_LOOPBACK`).

- **WSL1, or WSL2 in mirrored networking mode** (`networkingMode=mirrored` in
  `.wslconfig`) shares the Windows loopback, so the default works. Every
  process in every WSL distro then reaches the runner as a loopback caller,
  credential routes included, so use it only where you trust every WSL user
  and container on the machine.
- **WSL2 in NAT mode** (the default) cannot reach the runner, because the runner
  listens on no address WSL can reach. Switch to mirrored
  mode. Forwarding a Windows port to the runner is not a supported setup:
  every client that can reach the forwarded address arrives at the runner as
  `127.0.0.1`, with a local caller's access.

## Available Tools

### Health Check

| Tool | Description |
|------|-------------|
| `ui_health` | Check if qontinui-runner is accessible |

### Control Mode (Runner's Own UI)

| Tool | Description |
|------|-------------|
| `ui_snapshot` | Get complete UI state with all elements |
| `ui_discover` | Force element discovery/registration |
| `ui_get_element` | Get detailed info for a specific element |
| `ui_click` | Click an element by ID |
| `ui_type` | Type text into an input element |
| `ui_focus` | Focus an element |

### SDK Mode (External Apps)

| Tool | Description |
|------|-------------|
| `sdk_connect` | Connect to an SDK-integrated app by URL |
| `sdk_disconnect` | Disconnect from the current app |
| `sdk_status` | Check SDK connection status |
| `sdk_elements` | Get all elements from the connected app |
| `sdk_snapshot` | Get a full UI snapshot from the connected app |
| `sdk_click` | Click an element by ID |
| `sdk_type` | Type into an element by ID |

> **Note:** The MCP server also includes legacy `extension_*` tools for browser tab access via a Chrome extension. These are deprecated and will be removed in a future release. Use the SDK tools instead.

## Tool Results and Errors

A tool that could not do what was asked returns its result with `isError: true`,
as the MCP spec requires. Branch on that flag, not on the text. This covers:

- the runner or app being unreachable, or answering with a failure (the text
  carries a `[code=… status=…]` tag when a code or HTTP status is known);
- a failure wrapped in a success envelope: a failed assertion (`passed: false`),
  a nested `success: false`, a wait that timed out, or an action that reports it
  did not run, change or delete anything;
- a missing or bad argument, an unknown tool, or an unmet precondition;
- an exception raised inside the tool, returned as `Error: <exception>` with no
  `[code=…]` tag.

When a response lacks the field that carries its verdict (an assertion's
`passed`, an action's `executed`, `sdk_status`'s `connected`, `sdk_find`'s
`found`, a design diff's `hasSignificantChanges`, and so on), the result says
**UNKNOWN** and is flagged `isError: true`. It is never reported as a pass, a
"no" or a "not found". Inside an answer that is otherwise complete, a missing
detail is marked UNKNOWN in place without flagging the call, for example a form
or field with no `isValid`/`valid` reads `validity UNKNOWN`, never valid.

An answer that happens to be empty is not an error and returns `isError: false`:
an empty clipboard, zero matches, no saved bookmarks or baselines, or
`sdk_status` reporting that no app is connected. `sdk_status` is an error only
when the status request itself fails, and both texts begin "SDK not connected",
so read the flag.

The flag needs `mcp>=1.19.0`, which `pyproject.toml` requires. Earlier releases
replace the failure text with an SDK validation error.

## Usage Examples

### Inspect Runner UI

```
AI: Let me check what's on the runner's Settings page.

1. First, get a snapshot of the UI:
   ui_snapshot

2. Click the Settings button:
   ui_click element_id="sidebar-nav-item-settings"

3. Get another snapshot to see the new page:
   ui_snapshot
```

### Inspect an SDK-Integrated App

```
AI: Let me check the login form on localhost:3000.

1. Connect to the app:
   sdk_connect url="http://localhost:3000"

2. Get all elements:
   sdk_elements

3. Type into the email field:
   sdk_type element_id="email-input" text="test@example.com"

4. Click the submit button:
   sdk_click element_id="login-button"
```

## Element IDs

Elements in the runner's UI have registered IDs that follow patterns like:

- `sidebar-nav-item-{name}` - Sidebar navigation items
- `button-{action}` - Action buttons
- `input-{field}` - Input fields
- `dialog-{name}` - Dialog components

Use `ui_snapshot` to discover all available element IDs.

## Architecture

```
Claude/AI
    |  MCP Protocol
ui-bridge-mcp (this server)
    |  HTTP
qontinui-runner (port 9876)
    |-- /ui-bridge/control/* --> Runner's Tauri webview
    |-- /ui-bridge/sdk/*     --> SDK-integrated apps (direct HTTP)
```

## Development

```bash
# Install dependencies
poetry install

# Run tests
poetry run pytest

# Run linting
poetry run ruff check .
poetry run mypy --strict src tests

# Format code
poetry run ruff format src
```

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for full terms.