Skip to main content
Glama
zaakirio

Bellows

by zaakirio
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

Bellows

An MCP (Model Context Protocol) server that manages a local llama.cpp model fleet. It lets an AI agent (Claude Code, Claude Desktop, or any MCP client) discover GGUF models on disk, start and stop llama-server processes safely, smoke-test running servers, and query historical eval results from a crucible SQLite database.

Bellows is the piece that turns "a directory full of GGUF files and a llama-server binary" into something an agent can operate: it owns the process lifecycle, refuses to touch anything it did not start, and reports structured JSON for every operation. It is for people running a local model zoo who want their coding agent, not a terminal full of scripts, to be the operator. Headline numbers, measured on Apple M4 Pro 24 GB on 2026-07-07 (details in Measured numbers): 27/27 tests green, and the full start -> status -> smoke -> stop lifecycle for a 697 MiB model completes in 716 ms warm through real MCP tool calls.

Why MCP

Managing a local inference fleet is a tool-use problem, not a chat problem. The operations (scan disk, spawn a server, poll health, run prompts, aggregate eval rows) are exactly the shape MCP tools are designed for: typed inputs, structured outputs, and clear error semantics. Exposing them over MCP means any compliant client gets the whole workflow for free, instead of shelling out to ad-hoc scripts with no validation and no ownership tracking.

Related MCP server: llauncher

Architecture

┌──────────────┐  stdio / streamable HTTP  ┌───────────────────────────────┐
│  MCP client  │ ◄───────────────────────► │  bellows (Node 24, TS)        │
│ (Claude Code)│                           │                               │
└──────────────┘                           │  models.ts    fs scan         │
                                           │  supervisor.ts spawn/health   │──► llama-server (child procs)
                                           │  smoke.ts     OpenAI-compat   │──► http://127.0.0.1:<port>/v1
                                           │  evals.ts     node:sqlite RO  │──► crucible results.db
                                           └───────────────────────────────┘

  • src/index.ts parses CLI flags and picks the transport (stdio by default, streamable HTTP with --http).

  • src/server.ts registers the six tools and two resources on an McpServer from the official SDK.

  • src/supervisor.ts is the process owner: it spawns llama-server with an argv array (never a shell string), polls /health until ready, keeps a stderr tail for diagnostics, and only ever signals PIDs it spawned itself.

  • src/evals.ts opens the crucible database with node:sqlite in read-only mode and runs aggregate queries.

  • src/models.ts scans a directory tree for .gguf files, skips mmproj-* projector files, guesses the quant from the filename, and collapses sharded -00001-of-000NN sets into one entry.

Tool catalog

Tool

What it does

Key inputs

list_models

Recursive GGUF scan: name, path, size, quant guess, shard detection. Skips mmproj-*.

dir?

server_status

Lists every bellows-owned server (pid, model, uptime, health) and optionally probes any port.

port?

start_server

Spawns llama-server, waits for /health, records the pid. Refuses if the port is taken. Accepts a path or a fuzzy model name.

model, port?, ngl?, ctx?, readyTimeoutMs?

stop_server

SIGTERM then SIGKILL after a grace period. Only for servers bellows started.

port

smoke_test

Sends N chat prompts over the OpenAI-compatible API, reports responses, latency, tokens/second.

port, prompts?, maxTokens?, timeoutMs?

eval_history

Read-only crucible queries: list runs, per-category run summary (pass rates plus complied/hedged/refused tallies), or compare two runs.

action, model?, runId?, runA?, runB?

Resources: bellows://models (the models list as JSON) and bellows://eval-runs (all eval runs as JSON).

Setup

Requires Node 24+ (for built-in node:sqlite) and a built llama.cpp checkout.

npm install
npm run build

Configuration is three environment variables:

BELLOWS_MODELS_DIR=/path/to/models          # scanned recursively for .gguf
BELLOWS_LLAMA_SERVER=/path/to/llama-server  # the binary bellows spawns
BELLOWS_CRUCIBLE_DB=/path/to/results.db     # crucible eval db, opened read-only

Claude Code / Claude Desktop

{
  "mcpServers": {
    "bellows": {
      "command": "node",
      "args": ["/absolute/path/to/bellows/dist/index.js"],
      "env": {
        "BELLOWS_MODELS_DIR": "/Users/you/models",
        "BELLOWS_LLAMA_SERVER": "/Users/you/llama.cpp/build/bin/llama-server",
        "BELLOWS_CRUCIBLE_DB": "/Users/you/crucible/results.db"
      }
    }
  }
}

For Claude Code this goes in .mcp.json (project) or via claude mcp add; for Claude Desktop it goes in claude_desktop_config.json.

HTTP transport

node dist/index.js --http --port 8765

This exposes a stateless streamable-HTTP endpoint at POST http://127.0.0.1:8765/mcp for clients that prefer HTTP over stdio. --host changes the bind address (default 127.0.0.1); containers need --host 0.0.0.0 for published ports to work.

Docker

docker build -t bellows .
docker run --rm -p 8765:8765 \
  -v /path/to/models:/models -e BELLOWS_MODELS_DIR=/models \
  -v /path/to/results.db:/crucible/results.db:ro -e BELLOWS_CRUCIBLE_DB=/crucible/results.db \
  bellows

The image is multi-stage (node:24-slim, dev dependencies pruned), runs as the non-root node user, serves the HTTP transport on 0.0.0.0:8765, and has a container healthcheck that sends a real JSON-RPC initialize to /mcp. start_server is not useful inside the container unless you also bake in a llama-server binary and set BELLOWS_LLAMA_SERVER; the image is primarily for the scan/status/eval-history surface over HTTP.

Technical decisions

Transport: stdio primary, streamable HTTP opt-in. stdio is what Claude Code and Claude Desktop spawn natively, and it inherits the parent lifecycle, so servers bellows started die with the session instead of leaking. The HTTP transport is stateless (a fresh MCP server per request) but every request shares the one Supervisor instance, so process ownership survives across calls. Only POST is accepted; there is no session or SSE state to manage, which keeps the surface small.

Process ownership model. Bellows will only ever signal a ChildProcess handle it created itself. stop_server looks the port up in its own supervision table and errors if it is not there; there is no "kill whatever is on port X" path by design. Conversely start_server does a TCP probe first and refuses to bind a port where anything is already listening, so it can never fight another process for a port. Stop is graceful: SIGTERM, then SIGKILL only after a 10 s grace period. Bellows reaps all children before exiting, whether it receives SIGINT/SIGTERM or the MCP client simply closes stdin.

Why node:sqlite. The eval database is read-only from bellows' perspective, and Node 24 ships a synchronous SQLite driver in core. That removes a native-module dependency (better-sqlite3) and its rebuild churn for zero functional cost. The database is opened with readOnly: true, so bellows physically cannot write to crucible's data.

No shell, everywhere. llama-server is spawned via spawn(bin, argsArray); no string ever passes through a shell, so model paths with spaces or metacharacters are inert. All network calls (/health, /props, chat completions) carry AbortSignal.timeout deadlines. All tool inputs are zod-validated with ranges (ports 1024-65535 for binding, ctx up to 1M, at most 20 smoke prompts), and every failure path returns a message that says what to do next.

Testing

npm test              # everything
npm run test:unit     # scanning, DB queries, arg building, MCP schema surface
npm run test:integration  # real llama-server lifecycle (skips if binary/model missing)

Unit tests cover the quant/shard scanner against a synthetic directory tree, the crucible queries against the real results.db (read-only), argv construction, supervisor ownership refusals, and the full MCP tool/resource surface over an in-memory transport. The integration test starts the real LFM2.5-1.2B Q4_K_M model on port 8091 through the actual MCP tool calls, verifies a duplicate start is refused, smoke-tests it, stops it, and asserts the port is dead afterwards. It skips with a message when the llama-server binary, the models directory, or the model is absent.

Current results on the development machine (Apple M4 Pro, 24 GB, 2026-07-07): 26 unit tests pass, 1 integration test passes, 0 failures. In GitHub CI only the 26 unit tests exercise code; the integration test self-skips because the runner has no llama-server binary or model. CI also builds the Docker image on every push.

Transcript

Real tool calls against this machine, unedited except for trimming long model lists.

list_models {} found 6 models:

{
  "dir": "~/inf-eng/models",
  "count": 6,
  "models": [
    {
      "name": "LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M",
      "path": "~/inf-eng/models/LFM2.5-1.2B-Instruct-Uncensored-GGUF/LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M.gguf",
      "sizeBytes": 730895520,
      "sizeHuman": "697.0 MiB",
      "quant": "Q4_K_M",
      "sharded": false
    },
    {
      "name": "gemma-4-12b-it-uncensored-Q4_K_M",
      "path": "~/inf-eng/models/gemma-4-12b-it-uncensored-Q4_K_M.gguf",
      "sizeBytes": 7381381760,
      "sizeHuman": "6.87 GiB",
      "quant": "Q4_K_M",
      "sharded": false
    }
  ]
}

start_server {"model": "LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M", "port": 8091, "ctx": 2048}:

{
  "pid": 64539,
  "port": 8091,
  "modelPath": "~/inf-eng/models/LFM2.5-1.2B-Instruct-Uncensored-GGUF/LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M.gguf",
  "startedAt": "2026-07-07T20:29:22.805Z",
  "apiBase": "http://127.0.0.1:8091/v1"
}

A second start_server on the same port was refused with an error naming the existing pid and model.

server_status {"port": 8091}:

{
  "ownedServers": [
    {
      "pid": 64539,
      "port": 8091,
      "modelPath": ".../LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M.gguf",
      "startedAt": "2026-07-07T20:29:22.805Z",
      "uptimeSeconds": 1,
      "alive": true,
      "healthy": true
    }
  ],
  "probe": { "port": 8091, "healthy": true, "ownedByBellows": true, "model": ".../LFM2.5-1.2B-Instruct-Uncensored-Q4_K_M.gguf" }
}

smoke_test {"port": 8091}:

{
  "port": 8091,
  "prompts": 3,
  "results": [
    { "prompt": "Reply with exactly one word: pong", "response": "pong", "latencyMs": 59, "completionTokens": 3, "tokPerSec": 320.7 },
    { "prompt": "What is 17 * 23? Answer with just the number.", "response": "391", "latencyMs": 40, "completionTokens": 2, "tokPerSec": 395.7 },
    { "prompt": "Name the capital of France in one word.", "response": "Paris", "latencyMs": 41, "completionTokens": 2, "tokPerSec": 373.4 }
  ],
  "meanLatencyMs": 47,
  "meanTokPerSec": 363.3
}

stop_server {"port": 8091}:

{ "port": 8091, "pid": 64539, "exitCode": 0, "forced": false }

eval_history {"action": "compare_runs", "runA": 23, "runB": 24} (base vs abliterated LFM2.5 Q4_K_M, falsereject category excerpt):

{
  "category": "falsereject",
  "a": { "n": 50, "complied": 7, "hedged": 43, "refused": 0 },
  "b": { "n": 50, "complied": 39, "hedged": 11, "refused": 0 }
}

Measured numbers

  • Model count on this machine: 6 runnable GGUFs (mmproj files excluded).

  • The full start -> status -> smoke -> stop lifecycle for LFM2.5-1.2B Q4_K_M (697 MiB) completed in 716 ms in the integration test, including model load to first healthy /health response, with the model file warm in the page cache.

  • Smoke-test mean latency was 47 ms per request over 2-3 token completions.

  • The tokens/second figures (320-396) are llama.cpp's own timings.predicted_per_second over those tiny generations. Crucible's eval history for the same model reports roughly 60 tok/s sustained over full-length responses.

  • The HTTP transport was verified with raw JSON-RPC initialize and tools/list requests via curl.

Available Tools

6 tools
eval_historyQuery crucible eval historyA

Read-only queries against the crucible eval results database: list runs for a model, summarize one run per category (pass rates plus complied/hedged/refused tallies), or compare two runs.

ParametersJSON Schema
NameRequiredDescriptionDefault
runANoFor compare_runs: baseline run id.
runBNoFor compare_runs: comparison run id.
modelNoFor list_runs: substring filter on model name or file.
runIdNoFor run_summary: the run id.
actionYeslist_runs: all eval runs (optionally filtered by model). run_summary: per-category pass rates and complied/hedged/refused tallies for one run (requires runId). compare_runs: side-by-side category deltas (requires runA and runB).

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explicitly labels the tool as 'Read-only queries', indicating no destructive side effects. It describes the output for each action, but lacks details on potential rate limits or authentication requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, two sentences that front-load the essential information and then list the actions without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters (1 required) and no output schema, the description provides sufficient context for the three actions. It covers what each action returns, though it could elaborate on the output format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with detailed parameter descriptions. The tool description adds overall context but does not significantly enhance the meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it is a read-only query tool for crucible eval results, listing three specific actions (list runs, summarize, compare). This distinguishes it from sibling tools like list_models and server_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description enumerates the three actions with brief explanations, providing context for when to use each. However, it does not explicitly state when not to use this tool or compare with alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_modelsList local GGUF modelsA

Scan a directory recursively for runnable .gguf model files (skipping mmproj-* projector files). Returns name, path, size, a quant guess from the filename, and sharded-set detection.

ParametersJSON Schema
NameRequiredDescriptionDefault
dirNoDirectory to scan recursively for .gguf files. Defaults to BELLOWS_MODELS_DIR.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dirYes
countYes
modelsYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description adequately discloses scanning behavior, file filtering, and return fields. However, it does not mention potential performance impacts or restrictions like recursion depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, informative sentence that front-loads key information with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists, the description covers return values adequately. It could mention that results are returned as an array, but is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds that the directory defaults to BELLOWS_MODELS_DIR and that scanning is recursive, providing value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool scans a directory recursively for .gguf model files, skips projector files, and returns specific fields. It is distinct from sibling tools like start_server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for finding local models, but no explicit when-to-use or when-not-to-use guidance relative to siblings. Siblings are different enough that confusion is unlikely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

server_statusllama-server statusA

Report all llama-server processes bellows is supervising (pid, port, model, uptime) and optionally probe a specific port's /health endpoint, including servers bellows does not own.

ParametersJSON Schema
NameRequiredDescriptionDefault
portNoAlso probe this port for a llama-server health endpoint, owned by bellows or not.

Output Schema

ParametersJSON Schema
NameRequiredDescription
probeNo
ownedServersYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that the tool reports on supervised processes and can probe a health endpoint for any server, including those not owned by bellows. Since no annotations are provided, the description adequately conveys the read-only nature and scope.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the core purpose and immediately provides key details. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single optional parameter and the presence of an output schema, the description fully covers what the tool does, including the types of information reported (pid, port, model, uptime) and the optional probe behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds no new information about the 'port' parameter beyond what is already in the schema description (probing a health endpoint). With 100% schema coverage, the baseline is 3, and the description does not elevate it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool reports supervised llama-server processes with specific details (pid, port, model, uptime) and optionally probes a health endpoint. It distinguishes itself from sibling tools like start_server and stop_server by focusing on status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

While the description does not explicitly state when to use this tool over alternatives, the sibling names and the tool's purpose imply it is for obtaining server status rather than starting/stopping. A slight improvement would be to include a direct usage note.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smoke_testSmoke-test a running serverA

Send a batch of chat prompts to a running llama-server over its OpenAI-compatible API and report each response with latency and generation speed (tokens/second as measured by llama.cpp).

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort of a running llama-server.
promptsNoPrompts to send. Defaults to three short sanity prompts.
maxTokensNomax_tokens per completion.
timeoutMsNoPer-request timeout in milliseconds.

Output Schema

ParametersJSON Schema
NameRequiredDescription
portYes
promptsYes
resultsYes
meanLatencyMsYes
meanTokPerSecYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that it sends chat prompts and reports latency/speed, but does not clarify side effects (e.g., whether it is read-only or modifies server state), auth requirements, or rate limits. The mention of 'as measured by llama.cpp' adds some context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-formed sentence that front-loads the action and resource. Every word contributes to meaning, resulting in zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, the description need not detail return values. It adequately covers the tool's purpose, inputs (implied by schema), and reported outputs (latency, speed). Prerequisites (running server) are obvious from context. Nearly complete for a test tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so each parameter already has a description. The tool description adds limited extra semantic value beyond the schema, only mentioning 'batched chat prompts' and output metrics. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool sends batch chat prompts to a running llama-server via OpenAI-compatible API and reports each response with latency and tokens/second. It uses a specific verb ('smoke-test') and resource ('running server'), distinguishing it from sibling tools like start_server or stop_server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for testing a running server but does not provide explicit when-to-use or when-not-to-use guidance. No alternatives or exclusions are mentioned, making it minimally adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_serverStart a llama-serverA

Spawn llama-server for a model (full arg validation, no shell), wait for its /health endpoint to report ready, and record the pid under bellows supervision. Refuses to start if the port is already in use.

ParametersJSON Schema
NameRequiredDescriptionDefault
ctxNoContext window in tokens.
nglNoGPU layers to offload (99 = all).
portNoHTTP port for llama-server.
modelYesModel to serve: an absolute path to a .gguf file, or a substring matched against the scanned models list (e.g. 'lfm2.5' or 'Q4_K_M').
readyTimeoutMsNoHow long to wait for the /health endpoint before giving up and killing the process.

Output Schema

ParametersJSON Schema
NameRequiredDescription
pidYes
portYes
apiBaseYes
modelPathYes
startedAtYes

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully discloses behavior: full arg validation, no shell, health wait, pid recording, port conflict refusal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with action and key constraints, no excess words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists, the description sufficiently explains the tool's operation and constraints for a 5-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so definitions exist; description adds 'full arg validation' but no additional parameter meaning beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the verb 'spawn' and specifies the resource 'llama-server for a model', clearly distinguishing it from sibling tools like stop_server and server_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or when-not-to-use guidance is given. The description implies usage for starting a model server but does not compare to alternatives like smoke_test.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_serverStop a bellows-owned llama-serverA

Gracefully stop (SIGTERM, then SIGKILL after a grace period) a llama-server that bellows started. Refuses to touch processes it does not own.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort of the bellows-owned server to stop. Bellows refuses to stop servers it did not start.

Output Schema

ParametersJSON Schema
NameRequiredDescription
pidYes
portYes
forcedYes
exitCodeYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the graceful stop strategy (SIGTERM then SIGKILL) and the ownership constraint, which are critical behavioral traits. It does not detail the grace period length or error handling, but the key behaviors are transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, with only two sentences that front-load the action and method. Every word contributes meaning, and there is no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, single-parameter tool with an output schema (not shown but present), the description provides sufficient context: the stopping method, ownership constraint, and implicit usage hint. It could mention potential error conditions or prerequisites, but given the simplicity, it is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description does not add additional parameter-specific guidance beyond what the schema already provides (the port's ownership constraint is repeated in the schema). No extra value is added for the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('gracefully stop') and the specific resource ('llama-server that bellows started'), effectively distinguishing it from the sibling tool 'start_server'. The mention of SIGTERM and SIGKILL further specifies the behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implicitly guides usage by stating it 'refuses to touch processes it does not own', indicating when not to use it. However, it does not explicitly compare to related siblings like 'server_status' or 'eval_history', leaving room for slight ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.1.0
    • First observedeval_history
    • First observedlist_models
    • First observedserver_status
    • First observedsmoke_test
    • First observedstart_server
    • First observedstop_server
Deploy Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.2/5.0

Scored across 6 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: evaluating results, listing models, checking server status, testing, starting, and stopping servers. No overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent snake_case verb_noun pattern (e.g., list_models, start_server). No deviations or mixed conventions.

Tool Count5/5

6 tools is well-scoped for a server focused on managing llama-server instances. Each tool is essential and covers core lifecycle and utility tasks.

Completeness4/5

Core lifecycle (start/stop), monitoring (status), testing (smoke_test), model listing, and eval querying are covered. Missing a restart or update tool, but the set is largely complete for the domain.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers