llm-gateway

Local MCP gateway to LLM-powered tools, backed by an OpenAI-compatible LLM (LM Studio by default). Runs shell commands and compresses their output, extracts answers from large files, drives local opencode and Codex agents, and runs browser-automation tasks.

Run

cd ui && npm install && npm run build
OPENAI_BASE_URL=http://127.0.0.1:1234/v1 uv run llm-gateway

ui/dist is a gitignored build artifact; build it before starting from a fresh checkout. Example systemd user units are in contrib/systemd/, see below.

Serves over HTTP (streamable-http) at http://<MCP_HOST>:<MCP_PORT>/mcp.

Related MCP server: Termada

Security

The gateway has no authentication and its tools run arbitrary shell commands, read any file and drive agents. It is meant for loopback use only. See SECURITY.md for the threat model and how to report issues.

• Every HTTP route (MCP and dashboard) rejects requests whose Host is not loopback or listed in LLM_GATEWAY_ALLOWED_HOSTS (421), whose Origin is not allowed (403), and POSTs without Content-Type: application/json (400). This blocks DNS rebinding and cross-site requests from pages open in your browser.

• A non-loopback MCP_HOST (e.g. 0.0.0.0) is refused at startup unless LLM_GATEWAY_ALLOW_REMOTE=1. With it set, anyone who can reach the port can run commands. Prefer an SSH tunnel.

Tools

Tools are auto-registered from tools/ (every BaseTool subclass); the handle() docstring becomes the MCP tool description. cwd must be an absolute existing directory for opencode, browser and codex. Each agent tool prints Session ID: ... on the first output line; pass it back as session_id to continue that session.

Codex tool

codex connects to the locally installed, already-authenticated Codex app-server. It needs no API key in the gateway process. Ensure codex is on the service's PATH, its service user has completed Codex authentication, and codex-app-server.service is running. Each call opens a connection to the private app-server socket, runs with the on-request approval policy, and confines each turn to a workspace-write sandbox rooted at its supplied cwd. The dashboard approval card handles requested sandbox escapes, command, file-change, and extra-permission prompts.

Dashboard & live streaming

A run dashboard is served at / (same host:port as /mcp): a two-pane UI listing past runs, streaming live progress, answering permission prompts, and showing Claude / Codex subscription usage.

/usage never reads credential files. Codex windows come from the local Codex app-server (account/rateLimits/read). Claude windows are pushed by Claude Code's own status line, which receives rate_limits on stdin (Pro / Max, after the first response in a session). Add this to your status line script right after it reads stdin; until it reports, the Claude card shows "no data yet":

input=$(cat)
printf '%s' "$input" | python3 /path/to/llm-gateway/scripts/claude_usage_report.py >/dev/null 2>&1 &

Each SSE event is {run_id, seq, ts, type, ...}. Lifecycle events (produced by the hub): run_started {meta}, meta {meta}, run_finished {exit_code, timed_out}. Content events (produced by each tool, translated into this canonical shape, see events.py): line {text}, text {part_id?, text}, reasoning {part_id?, text}, tool_use {part_id?, tool, arg, status, output?, duration_ms?, screenshot_url?} (status is one of running / completed / error), step_start {part_id?}, step_finish {part_id?, tokens_total, cost?}, permission_ask {part_id?, request_id, title, status, reply?} (status is pending or resolved). Codex runs additionally publish codex_event {event_type, data} for every app-server notification and request; the dashboard shows these as expandable trace entries so unrecognized future protocol events are not silently lost.

UI dev server with API proxy to LLM_GATEWAY_URL (default http://127.0.0.1:1235):

cd ui && npm run dev

Planner plugin

plugins/planner is a Claude Code plugin that reviews plans from plan mode in the browser. Its PermissionRequest hook on ExitPlanMode posts the plan to the gateway (POST /plan), opens /plan/{id} for line annotations and approve / request changes, then long-polls GET /plan/{id}/decision. If the gateway is unreachable it fails open to the built-in approval prompt. See plugins/planner/README.md for install. The hook reaches the gateway at LLM_GATEWAY_URL (default http://127.0.0.1:1235).

Claude Code MCP config

claude mcp add --transport http llm-gateway http://127.0.0.1:1235/mcp

Or in config:

{
  "mcpServers": {
    "llm-gateway": {
      "type": "http",
      "url": "http://127.0.0.1:1235/mcp"
    }
  }
}

Tools are exposed to Claude as mcp__llm-gateway__<tool>, e.g. mcp__llm-gateway__run_command.

Tests

uv run python -m unittest discover -s tests

systemd user services

Example units live in contrib/systemd/:

• llm-gateway.service requires codex-app-server.service and wants opencode-serve.service.

• codex-app-server.service runs codex app-server --listen unix://%t/codex-app-server/app-server.sock.

• opencode-serve.service runs opencode serve on 127.0.0.1:4096.

They assume the repo is cloned to ~/llm-gateway, uv and codex are in ~/.local/bin and opencode is in ~/.opencode/bin. Adjust WorkingDirectory, ExecStart and PATH to match your setup.

cp contrib/systemd/*.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now codex-app-server.service opencode-serve.service llm-gateway.service
journalctl --user -u llm-gateway.service -f