# llm-gateway

Local MCP gateway to LLM-powered tools, backed by an OpenAI-compatible LLM
(LM Studio by default). Runs shell commands and compresses their output,
extracts answers from large files, drives local `opencode` and Codex agents,
and runs browser-automation tasks.

## Run

```bash
cd ui && npm install && npm run build
OPENAI_BASE_URL=http://127.0.0.1:1234/v1 uv run llm-gateway
```

`ui/dist` is a gitignored build artifact; build it before starting from a fresh
checkout. Example systemd user units are in `contrib/systemd/`, see below.

Serves over HTTP (streamable-http) at `http://<MCP_HOST>:<MCP_PORT>/mcp`.

| Env                          | Default                                    | Purpose                                                        |
|------------------------------|--------------------------------------------|----------------------------------------------------------------|
| `MCP_TRANSPORT`              | `streamable-http`                          | Transport: `streamable-http`, `sse`, or `stdio`.               |
| `MCP_HOST`                   | `127.0.0.1`                                | Bind address (HTTP transports).                                |
| `MCP_PORT`                   | `1235`                                     | Bind port (HTTP transports).                                   |
| `OPENAI_BASE_URL`            | `http://127.0.0.1:1234/v1`                 | OpenAI-compatible backend for compression.                     |
| `OPENAI_API_KEY`             | `lmstudio`                                 | Backend API key.                                               |
| `COMPRESSOR_MODEL`           | empty                                      | Model to use; empty lets the backend use its loaded model.     |
| `COMPRESSOR_MAX_INPUT_CHARS` | `100000`                                   | Input size cap before truncation.                              |
| `OPENCODE_URL`               | `http://127.0.0.1:4096`                    | `opencode serve` endpoint for `opencode` and `browser`.        |
| `CODEX_APP_SERVER_SOCKET`    | `$XDG_RUNTIME_DIR/codex-app-server/app-server.sock` | Codex app-server unix socket.                         |
| `CODEX_APP_SERVER_TRANSPORT` | `websocket`                                | Codex socket protocol: `websocket` or `jsonl`.                 |
| `LLM_GATEWAY_ALLOW_REMOTE`   | unset                                      | Set to `1` to allow a non-loopback `MCP_HOST`.                 |
| `LLM_GATEWAY_ALLOWED_HOSTS`  | empty                                      | Extra allowed `Host` values, comma-separated (e.g. `192.168.1.5:1235`). |

## Security

The gateway has no authentication and its tools run arbitrary shell commands,
read any file and drive agents. It is meant for loopback use only. See
[SECURITY.md](SECURITY.md) for the threat model and how to report issues.

- Every HTTP route (MCP and dashboard) rejects requests whose `Host` is not
  loopback or listed in `LLM_GATEWAY_ALLOWED_HOSTS` (421), whose `Origin` is not
  allowed (403), and POSTs without `Content-Type: application/json` (400). This
  blocks DNS rebinding and cross-site requests from pages open in your browser.
- A non-loopback `MCP_HOST` (e.g. `0.0.0.0`) is refused at startup unless
  `LLM_GATEWAY_ALLOW_REMOTE=1`. With it set, anyone who can reach the port can
  run commands. Prefer an SSH tunnel.

## Tools

| Tool                | Params                                               | Purpose                                                          |
|---------------------|------------------------------------------------------|------------------------------------------------------------------|
| `run_command`       | `command`, `prompt`, `timeout=90`, `cwd`             | Run a shell command, return LLM-compressed output.               |
| `extract_from_file` | `file_path`, `prompt`, `cwd`                         | Extract only the requested information from a file.              |
| `opencode`          | `prompt`, `cwd`, `session_id`, `timeout=3600`        | Run the local opencode agent.                                    |
| `browser`           | `task`, `url`, `cwd`, `session_id`, `timeout=3600`   | Browser task via opencode with Playwright MCP; pass/fail verdict. |
| `codex`             | `prompt`, `cwd`, `session_id`, `timeout=3600`        | Run Codex through the local app-server.                          |

Tools are auto-registered from `tools/` (every `BaseTool` subclass); the
`handle()` docstring becomes the MCP tool description. `cwd` must be an
absolute existing directory for `opencode`, `browser` and `codex`. Each agent
tool prints `Session ID: ...` on the first output line; pass it back as
`session_id` to continue that session.

## Codex tool

`codex` connects to the locally installed, already-authenticated Codex
app-server. It needs no API key in the gateway process. Ensure `codex` is on the
service's `PATH`, its service user has completed Codex authentication, and
`codex-app-server.service` is running. Each call opens a connection to the
private app-server socket, runs with the `on-request` approval policy, and
confines each turn to a `workspace-write` sandbox rooted at its supplied `cwd`.
The dashboard approval card handles requested sandbox escapes, command,
file-change, and extra-permission prompts.

## Dashboard & live streaming

A run dashboard is served at `/` (same host:port as `/mcp`): a two-pane UI
listing past runs, streaming live progress, answering permission prompts, and
showing Claude / Codex subscription usage.

| Endpoint                            | Purpose                                                                 |
|-------------------------------------|-------------------------------------------------------------------------|
| `GET /`                             | Dashboard UI.                                                           |
| `GET /runs`                         | JSON list of run summaries.                                             |
| `GET /events`                       | SSE firehose of all runs' events.                                       |
| `GET /events/{id}`                  | SSE stream for one run; replays buffered events, then tails.            |
| `POST /permission/{request_id}/reply` | Answer a pending permission prompt: `{reply: once\|always\|reject, message?}`. |
| `GET /usage`                        | Claude and Codex usage windows; Codex cached 5 min, `?refresh=1` forces a re-read. |
| `POST /usage/claude`                | Claude Code status line reports `{rate_limits}`, see below.             |
| `GET /screenshot?path=...`          | Serve a png/jpg/webp screenshot published by the `browser` tool since server start. |
| `GET /plan/{id}`                    | Planner plan review UI, see below.                                      |

`/usage` never reads credential files. Codex windows come from the local Codex
app-server (`account/rateLimits/read`). Claude windows are pushed by Claude
Code's own status line, which receives `rate_limits` on stdin (Pro / Max, after
the first response in a session). Add this to your status line script right
after it reads stdin; until it reports, the Claude card shows "no data yet":

```bash
input=$(cat)
printf '%s' "$input" | python3 /path/to/llm-gateway/scripts/claude_usage_report.py >/dev/null 2>&1 &
```

Each SSE event is `{run_id, seq, ts, type, ...}`. Lifecycle events (produced by
the hub): `run_started {meta}`, `meta {meta}`, `run_finished {exit_code,
timed_out}`. Content events (produced by each tool, translated into this
canonical shape, see `events.py`): `line {text}`, `text {part_id?, text}`,
`reasoning {part_id?, text}`, `tool_use {part_id?, tool, arg, status,
output?, duration_ms?, screenshot_url?}` (`status` is one of `running` /
`completed` / `error`), `step_start {part_id?}`, `step_finish {part_id?,
tokens_total, cost?}`, `permission_ask {part_id?, request_id, title, status,
reply?}` (`status` is `pending` or `resolved`). Codex runs additionally publish
`codex_event {event_type, data}` for every app-server notification and request;
the dashboard shows these as expandable trace entries so unrecognized future
protocol events are not silently lost.

UI dev server with API proxy to `LLM_GATEWAY_URL` (default `http://127.0.0.1:1235`):

```bash
cd ui && npm run dev
```

## Planner plugin

`plugins/planner` is a Claude Code plugin that reviews plans from plan mode in
the browser. Its `PermissionRequest` hook on `ExitPlanMode` posts the plan to
the gateway (`POST /plan`), opens `/plan/{id}` for line annotations and
approve / request changes, then long-polls `GET /plan/{id}/decision`. If the
gateway is unreachable it fails open to the built-in approval prompt. See
`plugins/planner/README.md` for install. The hook reaches the gateway at
`LLM_GATEWAY_URL` (default `http://127.0.0.1:1235`).

## Claude Code MCP config

```bash
claude mcp add --transport http llm-gateway http://127.0.0.1:1235/mcp
```

Or in config:

```json
{
  "mcpServers": {
    "llm-gateway": {
      "type": "http",
      "url": "http://127.0.0.1:1235/mcp"
    }
  }
}
```

Tools are exposed to Claude as `mcp__llm-gateway__<tool>`, e.g.
`mcp__llm-gateway__run_command`.

## Tests

```bash
uv run python -m unittest discover -s tests
```

## systemd user services

Example units live in `contrib/systemd/`:

- `llm-gateway.service` requires `codex-app-server.service` and wants
  `opencode-serve.service`.
- `codex-app-server.service` runs
  `codex app-server --listen unix://%t/codex-app-server/app-server.sock`.
- `opencode-serve.service` runs `opencode serve` on `127.0.0.1:4096`.

They assume the repo is cloned to `~/llm-gateway`, `uv` and `codex` are in
`~/.local/bin` and `opencode` is in `~/.opencode/bin`. Adjust
`WorkingDirectory`, `ExecStart` and `PATH` to match your setup.

```bash
cp contrib/systemd/*.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now codex-app-server.service opencode-serve.service llm-gateway.service
journalctl --user -u llm-gateway.service -f
```