Skip to main content
Glama

llauncher

An MCP-first launcher and management tool for llama.cpp llama-server instances. The MCP contract is the product; the HTTP Agent, llauncher CLI, and Streamlit UI are co-equal consumers of the same llauncher/operations/ service layer — three surfaces over one core, designed for both programmatic control (LLM agents, multi-node automation) and human operators.

Features

Core (llauncher/operations/)

The stateless service layer that every surface delegates to (ADR-LLNCH-008). Adding a verb here surfaces it across all four boundaries automatically.

  • Verbs: start, stop, swap, cancel, delete_model, list_orphans, validate_models

  • Pre-flight seams: model-health probe and VRAM estimation, attachable as optional callables on swap()

  • ADR-LLNCH-010 port discipline: every verb takes port as a required argument — no auto-allocation, no env-var fallback

MCP Server

Canonical surface for LLM agents and automation. Stdio transport; full read + mutate coverage of the core verbs.

  • Discovery: list_models, get_model_config

  • Lifecycle: start_server, stop_server, swap_server, cancel_server, server_status, get_server_logs, list_orphans

  • Configuration CRUD: add_model, update_model_config, delete_model, validate_config, validate_models

  • Telemetry & audit: server_metrics, server_slots (ADR-LLNCH-019), read_audit (#64)

HTTP Agent

Same verbs over REST for multi-node setups (ADR-LLNCH-009 hub-spoke). Port-keyed routes (/start/{port}, /swap/{port}, /stop/{port}, /cancel/{port}, /footer-context/{port}) plus /status, /models, /models/validate. Always token-protected via an X-Api-Key header — including on loopback, where the token is auto-generated rather than operator-supplied (ADR-LLNCH-003). A non-loopback bind additionally refuses to start without a pre-existing token. Unlike the agent, the MCP server and local CLI are in-process and tokenless. See docs/auth.md for the full token/auth model.

Streamlit UI

Web dashboard for human operators. Four tabs: Dashboard (read-only running view), Models (config CRUD + per-model start/stop/swap with explicit port picker), Nodes (peer registry), Audit (local audit-log tail).

CLI (llauncher)

Typer command-line surface, co-equal with MCP and UI. Subcommand groups: model (list, info, remove, validate), server (start, stop, swap, cancel, status), orphan (list), node (add, list, remove, status), config (path, validate); plus a top-level audit command. Rich tables for human output and --json on every group for scripting.

Configuration

  • Config Persistence: Store configurations in ~/.llauncher/config.json (single source of truth)

  • Validation: Model paths verified, port conflicts detected, blacklists enforced

Related MCP server: Harbor MCP Server

Installation

# Clone the repository
git clone https://github.com/shanevcantwell/llauncher
cd llauncher

# Install in development mode (includes UI)
pip install -e .

# Optional: Install test dependencies
pip install -e ".[test]"

Windows Notes

If you see warnings like WARNING: Ignoring invalid distribution ~ during install:

# Clean up corrupted site-packages and reinstall
cd github\llauncher
rmdir /s /q .venv
python -m venv .venv
\.venv\Scripts\activate
pip install -e .

Set LLAMA_SERVER_PATH before your first model load. The code default (~/.local/bin/llama-server) does not exist on Windows, so an unset LLAMA_SERVER_PATH fails the very first /start with Server binary not found: C:\Users\...\.local\bin\llama-server. Point it at your actual llama-server.exe:

  • Dev / scripts\run.bat usage: set LLAMA_SERVER_PATH in the project-root .env (template: .env.example), e.g. LLAMA_SERVER_PATH=C:\path\to\llama-server.exe.

  • Service install (scripts\windows\install.ps1): set it in %USERPROFILE%\.llauncher\agent.env (template: scripts/windows/agent.env.example) — required whenever the service runs under NSSM's default LocalSystem account, since that account's home does not resolve to your own profile. install.ps1 prints a reminder on every run if this is still unset.

Quick Start

Use the runner scripts for easiest setup:

The dashboard requires the local agent to be running. Start the agent first (in its own terminal), then the dashboard in a second terminal. The UI deliberately does not auto-spawn the agent — see ADR-LLNCH-009 and the "Why doesn't the UI start the agent for me?" expander rendered on the dashboard when the agent is down.

Scripts live in scripts/, run from the repo root:

Linux/macOS: install first with pip install --user -e . from the repo root (see Installation) — run.sh install is disabled, since it populated a repo-local .venv disconnected from your global commands (issue #154).

./scripts/run.sh agent       # Terminal 1: start agent in foreground
./scripts/run.sh ui          # Terminal 2: start dashboard (requires agent)
./scripts/run.sh stop        # Stop running agent
./scripts/run.sh discover    # List discovered launch scripts

Windows:

scripts\run.bat install      :: Set up virtual environment and install
scripts\run.bat agent        :: Terminal 1: start agent in foreground
scripts\run.bat ui           :: Terminal 2: start dashboard (requires agent)
scripts\run.bat stop         :: Stop running agent
scripts\run.bat discover     :: List discovered launch scripts

There is no agent-bg (detached/background start) on either platform — it was removed when it started contending with the systemd/NSSM-managed agent. For a persistent agent, use the service installers under "Running as a service" below instead of a background shell command.

Running as a service

For a persistent install that survives reboots and restarts on crash, the agent ships with installers for systemd (Linux, user-mode) and NSSM (Windows). See docs/operations/run-as-a-service.md.

The UI supports two postures — pick whichever fits how you work:

  • On demand: llauncher-ui (or ./scripts/run.sh ui) starts the dashboard in the foreground for the session; close the terminal and it's gone.

  • Per-operator systemd --user service (ADR-LLNCH-022): install with scripts/systemd/install-ui.sh for a unit that restarts on crash (Restart=on-failure) and logs to journald (journalctl --user -u llauncher-ui -f). See docs/operations/run-as-a-service.md for the full install steps.

Usage

MCP Server

Start the MCP server:

llauncher-mcp

Or configure in your MCP client (e.g., Claude Code):

{
  "mcpServers": {
    "llauncher": {
      "command": "llauncher-mcp",
      "args": []
    }
  }
}

Trust boundary (stdio only). The MCP server speaks the MCP stdio transport and has no authentication of its own — it implicitly trusts whatever process spawned it over the stdio pipe (typically your MCP client, e.g. Claude Desktop / Claude Code). There is no network listener for MCP. Vetting the MCP client you hand these tools to is the operator's responsibility; llauncher cannot distinguish a benign caller from a malicious one once the stdio pipe is open. See docs/plans/security-hardening-plan.md §2.2 (control C5) for the threat-model rationale.

Available MCP Tools

Tool

Description

list_models

List all configured models with current status (running/stopped)

get_model_config

Get full configuration details for a specific model

start_server

Start a llama-server instance on a given port (model_name + port required; ADR-LLNCH-010)

stop_server

Stop a running server by port number

swap_server

Atomically swap models on a port with rollback guarantee (ADR-LLNCH-011)

cancel_server

Cancel an in-flight start/swap on a port (ADR-LLNCH-014)

server_status

Get status summary of all running servers

get_server_logs

Fetch recent log lines from a running server

list_orphans

List unmanaged llama-server processes on the local node (ADR-LLNCH-015)

update_model_config

Update an existing model's configuration

validate_config

Validate a configuration without applying it

validate_models

Read-only weight-file validation (existence, GGUF magic, advisory VRAM/lockfile) (#475, ADR-LLNCH-027)

add_model

Add a new model configuration to the store

delete_model

Delete a model configuration (refuses if running; ADR-LLNCH-008 §4.1)

read_audit

Read recent audit-log entries on this node (ADR-LLNCH-008, #64)

server_metrics

Live inference telemetry for a running server — safe tier, no prompt text (ADR-LLNCH-019)

server_slots

Per-slot detail including prompt text — sensitive tier, granted separately from server_metrics (ADR-LLNCH-019)

Streamlit UI

Start the UI using the runner script (recommended):

Linux/macOS:

./scripts/run.sh ui

Windows:

scripts\run.bat ui

Bind to loopback (no built-in auth). Streamlit binds wherever the operator launches it; the default is loopback. The runner scripts launch with --server.address 127.0.0.1, and that is the recommended invocation for typical single-operator use. The dashboard itself has no built-in authentication — anything that can reach the port can drive every mutate path (start/stop servers, edit configs, manage nodes). Do not expose it beyond loopback without an operator-supplied gateway in front: Tailscale, an SSH tunnel, or a reverse proxy that enforces auth. Passing --server.address 0.0.0.0 (or a LAN IP) without one of those is equivalent to publishing an unauthenticated admin console on your network. See docs/plans/security-hardening-plan.md §2.8 (control C12) for the threat-model rationale.

Dashboard Tab

Read-only running view (no mutate verbs live here per M4 Slice 13 / #50). Status indicators (🟢 Running / ⚫ Stopped), uptime, and live log tail for each active server. Use the Models tab to start/stop/swap.

Models Tab

Config CRUD plus the per-model verb buttons. Add / edit / delete configurations and drive Start, Stop, Swap against the selected target node. Includes the explicit port picker (ui/components/port_picker.py) — ADR-LLNCH-010 requires the operator to choose the port at every call site; there is no auto-allocation or remembered default.

Nodes Tab

Peer registry for multi-node setups. Add / list / remove remote agent nodes, test connectivity, and observe status. The sidebar node_selector (ui/components/node_selector.py) chooses which node the Models tab acts against.

Audit Tab

Tails the local audit log at LAUNCHER_AUDIT_PATH (~/.llauncher/audit.jsonl by default). Read-only view of commanded vs. observed events. Remote-node audit access is deferred per #64.

CLI

The llauncher Typer CLI is a co-equal consumer of llauncher/operations/ alongside the MCP server, HTTP Agent, and Streamlit UI. Every group supports a --json / -j flag for machine-readable output; the default is a Rich-rendered color table for human use.

A global --state-dir option (before the subcommand) points a single invocation at a config/state directory other than the default, with precedence --state-dir > LAUNCHER_STATE_DIR env > ~/.llauncher:

llauncher --state-dir /var/lib/llauncher model list

This is the mechanism for a non-login/non-interactive caller (an automation harness, a service account) to read a shared multiuser state dir without exporting LAUNCHER_STATE_DIR or symlinking ~/.llauncher.

Subcommand groups:

# Model configurations
llauncher model list
llauncher model info mistral-7b
llauncher model remove mistral-7b    # config-only; refuses while running (#276)
llauncher model validate             # read-only weight-file check, all models (#475, ADR-LLNCH-027)
llauncher model validate mistral-7b --no-vram

# Server lifecycle — port is required on start (ADR-LLNCH-010)
llauncher server start mistral-7b --port 8081
llauncher server stop 8081
llauncher server swap mistral-7b --port 8081   # ADR-LLNCH-011 5-phase swap with rollback
llauncher server cancel 8081         # ADR-LLNCH-014: signals an in-flight start/swap
llauncher server status --json

# Orphans — unmanaged llama-server processes (ADR-LLNCH-015, read-only)
llauncher orphan list

# Remote nodes (ADR-LLNCH-009)
llauncher node add my-server --host 192.168.1.100 --port 8765
llauncher node list
llauncher node status --all
llauncher node remove my-server

# Configuration store
llauncher config path                # print path to config.json
llauncher config validate mistral-7b # schema-only round-trip, no filesystem touch

# Audit log (ADR-LLNCH-008, #338)
llauncher audit --limit 50 --action started --result success

Each group also accepts --help. The runner scripts (./scripts/run.sh agent, ./scripts/run.sh ui) remain the easiest way to launch the agent and dashboard; the CLI subcommands above act against an already-running stack.

Configuration

Create model configurations directly in ~/.llauncher/config.json. Configs can be managed via the UI or MCP tools.

Example config entry:

{
  "mistral": {
    "name": "mistral",
    "model_path": "/path/to/model.gguf",
    "mmproj_path": null,
    "n_gpu_layers": 255,
    "ctx_size": 131072,
    "parallel": 1,
    "metrics": true,
    "slots": false,
    "extra_args": "--threads 8 --flash-attn on --cache-type-k f32 --cache-type-v f32"
  }
}

Per ADR-LLNCH-026 (issue #477), ModelConfig is not a mirror of llama-server's argument schema: mmproj_path, n_gpu_layers, ctx_size, parallel, metrics, and slots are the only fields llauncher acts on directly. Every other llama-server flag (--threads, --flash-attn, --cache-type-k/-v, sampling params, etc.) is a verbatim extra_args passthrough, in the spelling from llama-server --help — no pydantic content validation. The llauncher-owned deny-list (--alias, -m/--model, --host/--port, --api-key, --metrics, --slots/--no-slots) is enforced once, at launch time, in core/process.py::build_command.

Per ADR-LLNCH-010, port is supplied at every call site (UI port picker, CLI --port, MCP port arg, HTTP /start/{port} route) and is not persisted in the config. Legacy default_port entries in config.json are silently dropped on load.

State Paths & Volume Mounts (Docker)

Per ADR-LLNCH-008, the lockfile directory and audit log are env-configurable so a container can mount host state as a volume — letting an in-container agent (e.g. pi-coding-agent) introspect the state of llauncher running on the host.

Two env-var families are both current, not a typo. This state-paths family below is single-L (LAUNCHER_*); the agent-identity family under Multi-Node Management → Deployment (LLAUNCHER_AGENT_HOST etc.) is double-L. The split is real — tracked open in #151; don't rename either family ahead of that issue landing.

Env var

Default

Holds

LAUNCHER_STATE_DIR

~/.llauncher

Base for every derived path below

LAUNCHER_RUN_DIR

$LAUNCHER_STATE_DIR/run

Per-server lockfiles ({port}.lock) and swap markers

LAUNCHER_AUDIT_PATH

$LAUNCHER_STATE_DIR/audit.jsonl

Append-only JSON Lines audit log

LAUNCHER_LOG_DIR

~/.llauncher/logs

Per-server log directory (append mode, ADR-LLNCH-013)

Precedence for each path is the explicit per-path var (LAUNCHER_RUN_DIR / LAUNCHER_AUDIT_PATH / LAUNCHER_LOG_DIR) > the LAUNCHER_STATE_DIR-derived default. With every var unset the paths are byte-identical to the legacy ~/.llauncher/* layout, so setting them is opt-in.

To let a container read the host's live llauncher state, mount the host paths in read-only and point the in-container env vars at the mount:

docker run \
  -v "$HOME/.llauncher/run:/host-llauncher/run:ro" \
  -v "$HOME/.llauncher/audit.jsonl:/host-llauncher/audit.jsonl:ro" \
  -e LAUNCHER_RUN_DIR=/host-llauncher/run \
  -e LAUNCHER_AUDIT_PATH=/host-llauncher/audit.jsonl \
  my-agent-image

Mount read-only (:ro) when the container only introspects; drop :ro if the containerized process is the one commanding llauncher and must write lockfiles/audit entries. The audit log is a single file, so bind-mount the file itself (not its parent dir) to avoid masking sibling state.

Change Management

llauncher includes validation rules to prevent problematic actions:

  • Port conflicts: Prevents starting models on ports already in use

  • Blacklisted ports: Default blacklist includes port 8080 (commonly used by other services)

  • Model whitelists: Optionally restrict which models can be started

  • Caller blacklists: Restrict which callers (UI, MCP, etc.) can perform actions

Versioning

vN (v1, v2, v3 …) denotes the architecture generation; 0.x denotes the semver release (currently 0.4.1a0 / v0.4.1-alpha — read the live number from pyproject.toml or the latest git tag, not this doc). They are independent axes and do not map to each other — see docs/VERSIONING.md.

Project Structure

llauncher/
├── pyproject.toml
├── llauncher/
│   ├── __init__.py
│   ├── __main__.py
│   ├── cli.py                  # Typer CLI (model/server/orphan/node/config groups)
│   ├── state.py                # Legacy LauncherState — eviction-compat hook (ADR-LLNCH-008)
│   ├── operations/             # Stateless service layer; MCP/HTTP/CLI/UI all delegate here (ADR-LLNCH-008)
│   │   ├── start.py
│   │   ├── stop.py
│   │   ├── swap.py             # ADR-LLNCH-011 five-phase swap with rollback
│   │   ├── delete.py
│   │   ├── orphan.py           # ADR-LLNCH-015 read-only orphan listing
│   │   └── preflight.py        # Model-health + VRAM seams
│   ├── agent/                  # HTTP agent (FastAPI, port-keyed routes per ADR-LLNCH-010)
│   │   ├── auth.py
│   │   ├── config.py
│   │   ├── footer_cache.py     # /footer-context/{port} TTL cache (ADR-LLNCH-012)
│   │   ├── middleware.py
│   │   ├── routing.py
│   │   └── server.py           # Lifespan handler reaps managed children on SIGTERM/SIGINT
│   ├── mcp_server/             # MCP server (stdio transport)
│   │   ├── server.py
│   │   └── tools/              # servers / models / config tool groups
│   ├── core/                   # Primitive substrate (no LauncherState)
│   │   ├── audit_log.py        # JSON Lines audit (ADR-LLNCH-008)
│   │   ├── config.py           # ConfigStore — single source of truth
│   │   ├── gpu.py              # GPU collector (ADR-LLNCH-006)
│   │   ├── lockfile.py         # Atomic O_EXCL per-port lockfiles
│   │   ├── log_rotation.py     # ADR-LLNCH-013 append + rotate
│   │   ├── marker.py           # In-flight swap/start marker (ADR-LLNCH-011/014)
│   │   ├── model_health.py     # Cache probe (ADR-LLNCH-005)
│   │   ├── process.py          # Subprocess management
│   │   └── settings.py         # LAUNCHER_* env-var family
│   ├── models/
│   │   └── config.py           # Pydantic ModelConfig (no default_port; ADR-LLNCH-010)
│   ├── remote/                 # Multi-node hub-spoke (ADR-LLNCH-009)
│   │   ├── node.py             # RemoteNode (port-keyed ops)
│   │   ├── registry.py         # NodeRegistry
│   │   └── state.py            # RemoteAggregator (swap_on_node parity)
│   └── ui/                     # Streamlit dashboard
│       ├── app.py
│       ├── utils.py            # render_op_result, OpResultSeverity ladder
│       ├── components/
│       │   ├── node_selector.py
│       │   └── port_picker.py  # Explicit port input — no auto-allocation
│       └── tabs/
│           ├── audit.py
│           ├── dashboard.py    # Read-only running view
│           ├── models.py       # Config CRUD + start/stop/swap verbs
│           └── nodes.py

Testing

Run the test suite:

pytest
# or with coverage
pytest --cov=llauncher --cov-report=term-missing

Running from a git worktree: the dev .venv is a shared editable install, so its .pth entry always resolves import llauncher to whichever checkout it was last pip install -e'd from (normally the main checkout) — a worktree invocation whose collection order lets that .pth win reads coverage against the main checkout's files, not the worktree's (#361). [tool.coverage.paths] in pyproject.toml reconciles the report, but the sanctioned invocation is still to pin the import explicitly:

PYTHONPATH="$(pwd)" pytest --cov=llauncher --cov-report=term-missing

Never repoint the shared venv's .pth at a worktree to work around this — that .venv also backs the live llauncher-agent systemd service, and a crash mid-window leaves the live service importing worktree code. Restore-after-use is not a substitute for not mutating it.

Test files are in tests/:

  • tests/unit/: Unit tests for models, config, and process

  • tests/integration/: Integration tests for state management

For an inventory of which tests exist (file-by-file, with markers and docstring first lines), see docs/generated/TEST_SUITE_SUMMARY.md. Regenerate after adding or renaming tests:

python scripts/summarize_tests.py

The coverage floor is pinned at --cov-fail-under=93 against non-UI scope in pytest.ini; UI coverage is deferred to the AppTest harness in #69 (v3-alpha).

Multi-Node Management (Remote)

llauncher supports managing llama-server instances across multiple machines (Windows and Linux) on a local network from a single dashboard.

Architecture

Each managed node runs a lightweight agent that exposes an HTTP API. The "head" dashboard connects to these agents over the LAN:

┌─────────────────────────────────────┐
│         HEAD DASHBOARD              │
│  - Streamlit UI with node selector  │
│  - Connects to all agents via HTTP  │
└─────────────┬───────────────────────┘
              │ LAN (port 8765)
    ┌─────────┼─────────┐
    ▼         ▼         ▼
┌────────┐ ┌────────┐ ┌────────┐
│ Agent  │ │ Agent  │ │ Agent  │
│ Linux  │ │Windows │ │ Linux  │
│ :8765  │ │ :8765  │ │ :8765  │
└────────┘ └────────┘ └────────┘

Deployment

1. Install on Each Node

On every machine you want to manage (including the head):

Linux/macOS:

git clone https://github.com/shanevcantwell/llauncher
cd llauncher
pip install --user -e .

Windows:

git clone https://github.com/shanevcantwell/llauncher
cd llauncher
scripts\run.bat install

2. Start the Agent on Each Node

Using runner scripts (recommended):

Linux/macOS:

./scripts/run.sh agent     # Foreground
./scripts/run.sh stop      # Stop agent

Windows:

scripts\run.bat agent      :: Foreground
scripts\run.bat stop       :: Stop agent

Neither script has a background/detached mode — see "Running as a service" above for a persistent agent that survives reboots.

With custom configuration:

# Linux/macOS
LLAUNCHER_AGENT_PORT=9000 LLAUNCHER_AGENT_NODE_NAME="my-server" ./scripts/run.sh agent

# Windows (PowerShell)
$env:LLAUNCHER_AGENT_PORT="9000"
$env:LLAUNCHER_AGENT_NODE_NAME="my-server"
scripts\run.bat agent

Environment Variables:

  • LLAUNCHER_AGENT_HOST: Host to bind to (default: 127.0.0.1). Set to 0.0.0.0 or a specific LAN IP to expose the agent to other hosts — see "Security Notes" below.

  • LLAUNCHER_AGENT_PORT: Port to listen on (default: 8765)

  • LLAUNCHER_AGENT_NODE_NAME: Friendly name for the node

  • LLAUNCHER_AGENT_TOKEN: The agent's X-Api-Key token. The agent always enforces a token (auth is never off, even on loopback); this var lets you supply it explicitly and always wins over the file below. Required when binding to anything other than loopback — the agent refuses to start on a non-loopback host without it. Special value - reads the token from stdin (one line). On a loopback start with no value set (and no LLAUNCHER_AGENT_TOKEN= line in agent.env), a fresh token is auto-generated and appended into ~/.llauncher/agent.env (mode 0600 if newly created). For the full token/auth model (which consumers need a token, exempt paths, resolution order), see docs/auth.md.

3. Start the Dashboard on the Head Machine

Linux/macOS:

./scripts/run.sh ui

Windows:

scripts\run.bat ui

The dashboard will automatically:

  1. Show a loading screen while initializing

  2. Register itself as the "local" node

4. Add Remote Nodes

In the dashboard:

  1. Go to the Nodes tab

  2. Click ➕ Add New Node

  3. Enter:

    • Node Name: Friendly name (e.g., linux-box, windows-server)

    • Host: IP address or hostname (e.g., 192.168.1.100)

    • Port: Agent port (default: 8765)

    • API Key: the remote agent's token (see Adding a remote node below)

  4. Click 🔍 Test Connection to verify

  5. Click ➕ Add Node to register

Adding a remote node (token walkthrough)

A remote agent always enforces a token (auth is never off, even on loopback). To pair the head with a remote node you copy that token by hand — it currently is not issued automatically (session-token issuance is tracked under #137). The token lives in a single live file per platform — agent.env, parsed directly by the agent and the local UI (issue #284):

Platform

Live source (agent.env)

Seeded (once) by

Linux

~/.config/llauncher/agent.env

scripts/systemd/install.sh

Windows

%USERPROFILE%\.llauncher\agent.env

scripts/windows/install.ps1

Step by step:

  1. Get on the remote box. SSH to a Linux node, or RDP to a Windows node.

  2. Read the token. The portable way is the agent's own subcommand, which resolves the token from env / stdin / agent.env and prints it to stdout:

    llauncher-agent print-token

    If you prefer to read the file directly, look for the LLAUNCHER_AGENT_TOKEN= line:

    # Linux
    grep LLAUNCHER_AGENT_TOKEN= ~/.config/llauncher/agent.env
    # Windows (PowerShell)
    Select-String LLAUNCHER_AGENT_TOKEN= $env:USERPROFILE\.llauncher\agent.env

    Over SSH you can do both in one shot: ssh windows-box llauncher-agent print-token.

  3. Copy the value. It is a single secrets.token_urlsafe(32) string on one line.

  4. Paste it into the head's UI. Back on the head machine, paste the value into the API Key field of the Add New Node form (step 3 above).

The token is stored on the head at ~/.llauncher/node_tokens.json (mode 0600); the local node is excluded because its token already lives in agent.env.

Network Configuration

Firewall Rules

Ensure port 8765 is open on managed nodes:

Linux (ufw):

sudo ufw allow 8765/tcp

Linux (firewalld):

sudo firewall-cmd --permanent --add-port=8765/tcp
sudo firewall-cmd --reload

Windows (PowerShell):

New-NetFirewallRule -DisplayName "llauncher Agent" -Direction Inbound -LocalPort 8765 -Protocol TCP -Action Allow

Security Notes

  • Loopback by default: The agent binds to 127.0.0.1 unless LLAUNCHER_AGENT_HOST is set explicitly. Set it to a LAN IP (or 0.0.0.0) to expose the agent to other hosts on the network.

  • Token required for non-loopback binds: Binding to anything other than 127.0.0.1 / ::1 / localhost requires LLAUNCHER_AGENT_TOKEN to be set. The agent refuses to start otherwise. On loopback first-run with no token configured, a fresh token is generated and appended into ~/.llauncher/agent.env (mode 0600 if newly created) and printed once to stderr.

  • Trusted LAN Only: Even with a token, only expose the agent on networks you trust — the transport is plain HTTP (no TLS). Tailscale is the recommended option for cross-host trust.

  • Firewall: Restrict port 8765 to your LAN subnet.

  • Full auth model: For the token/auth reference — the two planes (token-bearing HTTP agent vs. tokenless local MCP/CLI), the X-Api-Key header, exempt paths, resolution precedence, and file locations/modes — see docs/auth.md.

Usage

The sidebar Node Selector (ui/components/node_selector.py) picks the target node — local plus any registered remotes. A single target is always selected; the "All Nodes" cross-node aggregate view was dropped in M4 Slice 13 (#50).

  • Dashboard Tab: read-only running view across the selected node.

  • Models Tab: config CRUD + per-model Start / Stop / Swap, acting on the selected node.

  • Nodes Tab: registered-nodes list with Test Connection and Remove controls.

  • Audit Tab: tails the local LAUNCHER_AUDIT_PATH. Remote-node audit access is deferred per #64.

Troubleshooting

"Connection Failed" when adding node

  1. Verify agent is running on the remote node:

    curl http://<node-ip>:8765/health
  2. Check firewall rules on the remote node

  3. Verify the agent is binding to the correct interface:

    # Default is 127.0.0.1:8765 (loopback). For LAN access you must
    # have set LLAUNCHER_AGENT_HOST and LLAUNCHER_AGENT_TOKEN.
    netstat -tlnp | grep 8765

Agent won't start

  1. Check if port 8765 is already in use:

    lsof -i :8765
    # or
    netstat -tlnp | grep 8765
  2. Use a different port:

    LLAUNCHER_AGENT_PORT=9000 llauncher-agent

Can't connect from Windows to Linux (or vice versa)

  1. Verify network connectivity:

    ping <remote-node-ip>
  2. Check that the agent is not binding to loopback only:

    • The default is 127.0.0.1:8765. For cross-host access set LLAUNCHER_AGENT_HOST=0.0.0.0 (or a specific LAN IP) and LLAUNCHER_AGENT_TOKEN — the agent refuses to start on a non-loopback host without a token.

API Documentation

When an agent is running, visit http://<node-ip>:8765/docs for interactive API documentation.

License

MIT

Available Tools

15 tools
add_modelC

Add a new model configuration

ParametersJSON Schema
NameRequiredDescriptionDefault
configYesFull model configuration

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It only states 'add' but does not disclose write behavior, side effects (e.g., persistence, conflicts with existing configs), or required permissions. For a mutation tool, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short (one sentence), which might be considered concise, but it sacrifices necessary detail. It does not fully earn its place given the lack of content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations, no output schema, and a complex nested parameter (config with 10 subfields), the description provides no additional context about return values, required server state, or relationships between parameters. The description is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% per context, so baseline is 3. The description adds no meaning beyond the schema; it repeats 'Add a new model configuration' without explaining how parameters like metrics, no_mmap, or threads affect the operation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Add a new model configuration' uses a verb ('Add') and resource ('model configuration') but lacks specificity about what constitutes a model configuration and how it differs from updating or validating. It is clear in a minimal sense but vague.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus sibling tools like update_model_config or validate_config. No prerequisites or typical context (e.g., whether server must be running) are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancel_serverA

Cancel an in-flight start or swap on this port (ADR-014). Sets a cancel flag on the in-flight marker; the running op picks it up at the next phase boundary (typically within ~1 s during readiness poll). Returns success with marker_existed=False if there is no in-flight op — 'nothing to cancel' is a successful no-op, not an error. A cancel that arrives after the new process has been spawned and the lockfile written is ignored; the op completes and reports cancel_ignored_post_commit=True.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number of the in-flight op to cancel

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Fully discloses mechanism (cancel flag, phase boundary), timing (~1 s), and return conditions (marker_existed=False, cancel_ignored_post_commit=True), especially important since no annotations are provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, each sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given single parameter and no output schema, description covers behavior, edge cases, and timing comprehensively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers port parameter fully with description; description mentions it in context but doesn't add significant new semantics beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it cancels an in-flight start or swap on a port, with specific verb-resource pairing and reference to ADR-014, distinguishing it from sibling tools like start_server and stop_server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Describes when to use (cancel in-flight op) and explains edge cases (no-op if no in-flight op), but doesn't explicitly contrast with alternatives like stopping a server directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_modelA

Delete a model configuration. Refuses with action='rejected_in_use' (and the holding port) if the model is currently running anywhere; stop or swap first. Idempotent on a missing name (action='not_found').

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the model to delete

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses key behaviors: rejection with action code if in use, idempotency on missing name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with main action, no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, description covers edge cases (in-use and missing) sufficiently for a one-parameter delete tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema already describes the single parameter with 100% coverage. Description adds no additional meaning about the parameter itself beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action: delete a model configuration. It distinguishes from siblings by specifying behaviors for when the model is in use or missing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to stop or swap the model if it's running before deletion, providing clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_model_configA

Get the full configuration for a specific model by name

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the model to retrieve

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description only states 'get' without indicating read-only nature, permissions, or potential side effects. Minimal behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no unnecessary words. Front-loaded with key information: purpose and how to specify the model.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with one parameter and no output schema, the description covers the basics. Could mention that output is the full config object or that it is read-only, but still adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter. Description repeats the schema's 'Name of the model to retrieve' without adding extra context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states action (get), resource (full configuration), and identifier (by name). Distinguishes from sibling tools like list_models (listing) and update_model_config (modification).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like list_models or validate_config. Usage is implied but not clarified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_server_logsA

Fetch recent logs for a running server by port

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number of the server
linesNoNumber of log lines to fetch (default: 100)

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, placing the full burden on the description. It only states the basic action (fetch logs) and a qualifier ('recent'), without disclosing behavioral details such as whether logs are truncated, whether the server must be actively running, or how errors are handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that conveys the essential purpose without unnecessary words. It is front-loaded with the key action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description does not explain the return format or error states. While the tool is simple, the omission of response structure (e.g., array of log lines) and error conditions makes it less complete for an agent to anticipate the outcome.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all parameters (100% coverage) with clear descriptions for 'port' and 'lines'. The tool description adds no additional meaning beyond what the schema provides, meeting the baseline for this dimension.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Fetch'), resource ('logs for a running server'), and qualifier ('recent' and 'by port'). It clearly distinguishes from sibling tools like server_status or server_metrics, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. However, the tool is the only log-fetching tool among siblings, so confusion is minimal. The implication that the server must be running is present but not elaborated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_modelsA

List all configured models with their current status (running/stopped). Returns {models: [...], count: N} where each entry is nested as {identification: {name, model_path}, status: {state, port, pid?}}. The value to pass as model_name to start_server / swap_server is identification.name exactly as returned — do NOT concatenate it with status.port or any other field.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses return structure (status, nested fields) and says it lists 'all configured models'. No destructive behavior is expected, and this is adequately covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no waste. First sentence states purpose and return shape. Second sentence provides critical usage guidance. Both are essential.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter listing tool with no output schema or annotations, the description covers return format and usage. It lacks edge-case info (e.g., empty models) but is otherwise sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has zero parameters with schema coverage 100%. Baseline score 4 is appropriate; description adds no parameter info but does not need to.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses specific verb 'List all configured models with their current status', clearly identifying the tool's purpose. It distinguishes from siblings like start_server/swap_server by detailing return format.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells the agent how to use the output (pass identification.name to start_server/swap_server) and what not to do (do not concatenate with port). No explicit when-to-use vs alternatives, but context signals imply this is the listing tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_orphansA

List unmanaged llama-server processes on this node (ADR-015). An orphan is a live llama-server that llauncher did not launch — its (port, pid) does not match any live lockfile. Returns each orphan's pid, port (when discoverable from argv), and a cmdline_unreadable flag for processes whose argv could not be read. Adopt is intentionally not exposed in this revision — see ADR-015 §Deferred Work.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It explains the output fields (pid, port, cmdline_unreadable flag) and mentions that adopt is not exposed. This is good transparency for a listing tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first defines purpose, second details output and deferred work. No wasted words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explains the return fields sufficiently. The tool has 0 parameters and no complex behavior, so the description is complete for an agent to use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so baseline is 4. The description confirms no parameters are needed and the tool's use is straightforward.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists unmanaged llama-server processes (orphans) and specifies the condition for being an orphan. It distinguishes from sibling tools, none of which list orphans.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide explicit when-to-use or alternatives, but the purpose is clear enough that an agent can infer usage. However, it lacks guidance on when not to use or comparisons to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

server_metricsA

Live in-server inference telemetry for a running server (ADR-LLNCH-019): phase (idle/prompt/generating), gen_tok_s, prompt_tok_s, slot counts, started_at. Safe tier — no prompt text. Local-node only. Returns a degraded envelope ({'available': false, 'reason': 'loading'|'no-metrics-flag'|'unreachable'}) rather than erroring when the target server can't be read.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number of the server

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully discloses safety (no prompt text), locality (local-node only), error handling (degraded envelope instead of errors), and specific failure reasons (loading, no-metrics-flag, unreachable).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose, concise, and every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description adequately explains all aspects: inputs, outputs, error behavior, safety, and limits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'port' is fully covered by the schema description; the tool description adds no further semantic detail beyond context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns live in-server inference telemetry (phase, token rates, slot counts, started_at) for a running server, distinguishing it from sibling tools like server_status or get_server_logs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for real-time metrics on a local node without prompt text, but does not explicitly compare to siblings or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

server_slotsA

Per-slot detail for a running server, including prompt text (ADR-LLNCH-019). Sensitive tier — grant separately from server_metrics. Local-node only. Returns {'available': false, 'reason': 'slots_disabled'} when the server was not started with --slots.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number of the server

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes two important behaviors: it returns a specific object when the server was not started with --slots, and it is local-node only. Sensitive tier also disclosed. No annotation contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with purpose and key details. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter, the description covers purpose, return on edge case, scope, and sensitivity. Complete without needing output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema describes port parameter fully, and description does not add additional parameter-specific details beyond what schema provides. Baseline score of 3 appropriate given 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it provides per-slot detail for a running server, including prompt text. It distinguishes from sibling tools like server_metrics and server_status by focusing on individual slots.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Mentions sensitive tier and local-node only, implying grant separately from server_metrics and limited scope. However, it does not explicitly state when to use this vs alternatives like server_metrics.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

server_statusA

Get the status of all running llama-servers

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral traits. It does not disclose whether the operation is read-only, safe, or requires permissions. It only states the basic action without side-effect or safety information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no redundancy, front-loaded with the core intent. Every word contributes meaning, and the description is precisely as long as necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description could benefit from mentioning what 'status' includes (e.g., uptime, health, number of servers). However, for a parameterless tool with clear scope, the description is minimally adequate but missing return value details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has zero parameters and schema coverage is 100%, so baseline is 4. The description adds no parameter information, but none is needed. It correctly describes the output scope without ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get the status of all running llama-servers' uses a specific verb and resource, clearly distinguishing it from sibling tools like server_metrics (specific metrics) and server_slots (specific slots). It precisely states the scope ('all running llama-servers').

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus siblings like server_metrics or server_slots. No mention of prerequisites, frequency, or alternatives. The description only states functionality without contextual usage advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_serverA

Start a model on an empty port. Fails with action='rejected_occupied' if a different model is already running on that port — use swap_server for that case. Both 'model_name' and 'port' are required; the port is always specified by the caller (ADR-010). The model_name must exactly match a model from list_models.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort to start the model on
model_nameYesName of the model to start

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, description reveals fail condition and error action value ('rejected_occupied'), adding context beyond schema. Lacks detail on success behavior or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each serving a distinct purpose: action, fail case with alternative, and parameter requirements. No redundant text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers fail condition, required params, port specification rule, and relationship to sibling tool. No output schema needed; description is sufficient for a start operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description adds that both parameters are required (already in schema) and that model_name must match a model from list_models, providing extra constraint.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action ('Start a model on an empty port'), verb 'start', resource 'model', and distinguishes from sibling 'swap_server'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to avoid this tool (if a different model is running on the port) and directs to 'swap_server'. Also notes required parameters and that model_name must match 'list_models'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_serverA

Stop whatever is running on this port. Idempotent: returns success with action='already_empty' if nothing was there. Returns action='stopped' on a successful termination.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number of the server to stop

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but description discloses idempotent behavior and two return actions. However, lacks details on termination method (graceful/kill) or effect on pending requests.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences front-loading the action and key behaviors. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, but description covers return actions and idempotency. Missing error conditions or permission requirements, but acceptable for a simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter with schema coverage 100%. The description adds no extra meaning beyond the schema's 'Port number of the server to stop'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb 'stop' targeting a port resource. Idempotency and distinct return actions are specified. Differentiates from siblings like 'start_server' and 'cancel_server'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies use when a server on a port needs stopping. No explicit guidance on when to use over alternatives like 'cancel_server' or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swap_serverA

Replace the model on this port with a different one. Primary use: an agent replacing its own brain on the harness's expected port. Performs the 5-phase swap (pre-flight, marker, stop, start, readiness) with rollback to the previous model on failure. Calling with the model already running on the port is a successful no-op (action='already_running'). Fails with action='rejected_empty' if the port is empty — use start_server for that case.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort number to swap the model on
model_nameYesName of the new model to start

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It details the 5-phase swap process and rollback on failure, and documents the no-op and rejection scenarios. However, it does not mention potential side effects or permissions required, but still provides substantial transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (3 sentences) and front-loaded with the primary purpose. Every sentence adds value: purpose, process overview, edge cases. No redundant or vague statements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 2 required parameters, no output schema, and no annotations, the description covers the primary use, the swap process, rollback behavior, and all documented edge cases (no-op, rejected_empty). It is fully adequate for the agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is high. The description adds meaning by explaining the context around the parameters (e.g., 'harness's expected port') and special values like 'already_running'. This goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a clear verb and resource: 'Replace the model on this port with a different one.' It distinguishes the tool from siblings like start_server by explicitly contrasting the failure case 'rejected_empty' and directing to use start_server for that scenario.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies the primary use case ('an agent replacing its own brain on the harness's expected port'), provides explicit when-not-to-use via the rejected_empty case, and names the alternative tool (start_server). It also describes behavior when the model is already running (successful no-op).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_model_configB

Update an existing model's configuration

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the model to update
configYesUpdated configuration fields

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does not disclose behavioral traits such as whether the update triggers a server restart, whether the model remains active during update, or any error conditions. This is insufficient for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is concise and front-loaded. Every word contributes to clarity with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of the nested config object and the presence of many sibling tools, the description lacks information about the return value, side effects, or server state dependencies. It is not sufficiently complete for an agent to use safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters and nested fields. The tool description adds no additional meaning beyond what is in the schema, meeting the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb 'Update' and resource 'existing model's configuration', distinguishing it from siblings like add_model and get_model_config. It is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description provides no explicit guidance on when to use this tool versus alternatives like add_model, nor does it mention prerequisites (e.g., model must exist). However, the verb 'Update' implicitly suggests it modifies an existing resource, which is minimally adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_configC

Validate a model configuration without applying it

ParametersJSON Schema
NameRequiredDescriptionDefault
configYesConfiguration to validate

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description only mentions validation without applying, but fails to disclose whether it makes any changes, what it returns (e.g., success/failure, error details), or side effects. No annotations exist to mitigate this gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, concise sentence with no extraneous words. However, it could be slightly expanded to include key behavioral details without losing efficiency.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (nested object parameter, no output schema, no annotations), the description is insufficient. It does not explain validation behavior, error handling, or return format, making it hard for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'config' has a trivial description ('Configuration to validate') that adds no meaning beyond its name. Nested properties (name, ctx_size, etc.) lack individual descriptions, leaving the agent to guess their roles. Schema coverage is 100% but the description adds minimal value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates a model configuration without applying it, distinguishing it from siblings like update_model_config that modify configurations. However, it could explicitly differentiate more.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like get_model_config (to retrieve) or update_model_config (to apply). The agent is left to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.1.0
    • First observedadd_model
    • First observedcancel_server
    • First observeddelete_model
    • First observedget_model_config
    • First observedget_server_logs
    • First observedlist_models
    • First observedlist_orphans
    • First observedserver_metrics
    • First observedserver_slots
    • First observedserver_status
    • First observedstart_server
    • First observedstop_server
    • First observedswap_server
    • First observedupdate_model_config
    • First observedvalidate_config

TDQS

A3.9/5.0

Scored across 15 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: model CRUD, server lifecycle (start/stop/swap/cancel), monitoring (status/logs/metrics/slots), and orphan detection. Even closely related tools like server_metrics and server_slots are well-differentiated by their descriptions and sensitivity levels.

Naming Consistency5/5

All tools follow a consistent verb_noun pattern using lowercase snake_case (e.g., add_model, start_server, list_orphans). The naming is predictable and intuitive, with no mixed conventions.

Tool Count5/5

With 15 tools, the server covers model configuration, server lifecycle, monitoring, and utility functions like validation. The count is well-scoped for the domain—not too sparse to leave gaps, nor too bloated with redundant tools.

Completeness5/5

The tool surface provides complete CRUD for models, full server lifecycle (start, stop, swap, cancel), comprehensive monitoring (status, logs, metrics, slots, orphans), and validation. No obvious gaps prevent an agent from managing llama-server processes effectively.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI agents to discover, configure, and manage local development servers. Provides tools for app registration, port allocation, lifecycle control, and log access without manual config editing.
    232 npm
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Manages a local fleet of llama.cpp GGUF models, enabling AI agents to discover, start, stop, test, and query evaluation results for llama-server processes.
    6
    9 npm
    MIT
  • F
    license
    A
    quality
    C
    maintenance
    MCP server that connects LLM agents to a local LM Studio instance, enabling model management, OpenAI-compatible chat completions, text completions, and embeddings through a set of tools.
    9
    1
    -