stack-guardian
Collects NVIDIA GPU state as part of the stack snapshot and raises findings when a GPU is unavailable or when VRAM is used without an attributable loaded model.
Checks Ollama endpoints live: detects unreachable Ollama servers, models not loaded, models whose keep-alive is about to expire, and slow time-to-first-token. Also exposes a typed unload_model fix that POSTs keep_alive: 0 to an Ollama /api/generate to free a stuck or expiring model.
Reads OpenAI-compatible servers (LM Studio, vLLM, llama.cpp server) declared with "kind": "openai" via GET /v1/models, mapping their models into the same snapshot shape as an Ollama endpoint so the same loading and health rules apply.
Exposes stack health in Prometheus text format at GET /metrics, publishing per-finding gauges (code, severity, node), an overall worst-severity gauge, and the last run timestamp for scraping and alerting.
stack-guardian
mcp-name: io.github.LuminariSoftwares/stack-guardian
A health doctor for a home AI stack. It collects a snapshot of your nodes (Ollama endpoints, loaded models, NVIDIA
GPUs, tailnet peers), runs small Python rules against it, remembers the last runs, and reports findings with a
confidence grade: one sample is at most LIKELY, the same finding twice in a row is VERIFIED. A finding can carry a
typed fix that you run on purpose, never automatically.
Status: 0.1.0, first release. Standard library only, Python 3.11+.
Install
pip install stack-guardianThis gives you three commands: stack-guardian (the CLI), stack-guardian-serve and stack-guardian-mcp.
Try it offline first: the example snapshot makes every shipped rule fire once.
curl -LO https://raw.githubusercontent.com/LuminariSoftwares/stack-guardian/main/examples/snapshot.json
curl -LO https://raw.githubusercontent.com/LuminariSoftwares/stack-guardian/main/examples/ctx.json
stack-guardian doctor --snapshot snapshot.json --ctx ctx.jsonFrom a clone instead (for writing rules or running the tests):
git clone https://github.com/LuminariSoftwares/stack-guardian.git
cd stack-guardian
python -m venv .venv
.venv\Scripts\activate # Windows; macOS/Linux: source .venv/bin/activate
pip install -e .
stack-guardian doctor --snapshot examples/snapshot.json --ctx examples/ctx.jsonRelated MCP server: HomeLab Monitor
For coding agents
MCP server (any MCP client):
stack-guardian-mcpspeaks MCP over stdio with three read-only tools,stack_health,stack_snapshotandexplain. In Claude Code:claude mcp add stack-guardian -e STACK_GUARDIAN_NODES=/path/to/nodes.json -- stack-guardian-mcpOther clients: command
stack-guardian-mcp, args["--nodes", "/path/to/nodes.json"].stack_healthgrades against the saved run history without adding to it, and reuses one snapshot for 10 s (--ttl SECONDS).DSH plugin (
npm install dsh-stack-guardian, source indsh/, Node 20+): adds one line at the start of a DeepSeek Harness session when the doctor reports WARN or FAIL, and nothing otherwise. It runsstack-guardian doctor --json --quiet, so setSTACK_GUARDIAN_NODESto your nodes file in the harness environment.Watching and scraping:
stack-guardian doctor --watch 30 --jsonprints one JSON object per finding that changes state (new, resolved, severity change), so an agent can tail it;stack-guardian-serve --nodes FILEalso answersGET /metricsin Prometheus text format.
What it does
1. doctor - run the checks
Describe your nodes in a JSON file (see examples/nodes.json) and check them live:
stack-guardian doctor --nodes examples/nodes.json --ctx examples/ctx.jsonWith neither --snapshot nor --nodes, doctor reads the nodes file named by $STACK_GUARDIAN_NODES.
Options: --checks-dir DIR (your own rules, default is the shipped checks.d), --ctx FILE (rule settings such as
the model each node is expected to hold), --state FILE (default $STACK_GUARDIAN_STATE or
~/.stack-guardian/state.json), --json, --quiet (print nothing when all is OK).
Exit codes: 0 all OK/INFO, 1 a WARN, 2 a FAIL, 3 usage, 4 the doctor itself crashed.
Shipped rules: node.unreachable, ollama.unreachable, model.not_loaded, model.expiring, model.slow_ttft,
vram.unattributed, gpu.unavailable, tailnet.unavailable, tailnet.peer_offline, tailnet.relayed. Each raises one finding per affected node. A rule that
raises becomes a check.crashed WARN - a buggy rule never reports a broken stack.
LM Studio, vLLM, llama.cpp server. Give the node "kind": "openai" and list its base URLs under endpoints.
It is read from GET /v1/models (and GET /v1/models/<id> when the server has it) into the same shape as an Ollama
endpoint, so every rule above applies; a model a server lists as unloaded does not count as loaded.
{"gpu-box": {"kind": "openai", "endpoints": ["http://127.0.0.1:1234"]}}model.expiring (WARN): a loaded Ollama model whose /api/ps expires_at is within 60 s of the snapshot time;
INFO when /api/ps is unavailable. Change the window with "model_expiring_s": 120 in --ctx.
--watch N re-runs the doctor every N seconds and prints only the findings that changed state, each with a
timestamp. Ctrl+C exits 0 with a one-line summary. With --json, each change is one JSON object on stdout.
stack-guardian doctor --nodes examples/nodes.json --watch 30--ttft sends a 1-token generation to every loaded model on each Ollama endpoint and records ttft_ms on the
model in the snapshot. It costs a generation, so it never runs unless you pass it, and it needs --nodes. The probe
keeps each model's unload time as it was. model.slow_ttft (INFO) fires above 2000 ms; change it with
"ttft_slow_ms": 1500 in --ctx.
stack-guardian doctor --nodes examples/nodes.json --ttft2. doctor --fix ID - run a finding's typed fix
A rule may attach fix={"action": ..., "args": {...}} to a finding. There are exactly three actions and never a raw
shell string:
action | what it does | needs |
| POST |
|
| runs the argv list you declared for that lane |
|
| writes |
|
stack-guardian doctor --checks-dir examples/checks.d --snapshot examples/snapshot.json --fix ollama.stuck --config examples/lanes.jsonThat is a dry run: it prints the plan and changes nothing. Add --apply to perform it. An applied fix first saves
the snapshot under <state folder>/snapshots/, then runs, then appends one JSON line to <state folder>/fixes.jsonl
(--audit FILE to move it). A set_env value is never written to the audit, only its length. ID is the finding
code, or code@node when several nodes raise the same code. With --fix, the exit code is the fix's: 0 plan or
success, 1 refused or failed, 3 usage.
3. Other commands
stack-guardian snapshot --nodes FILE [--ttft]prints the raw snapshot the rules read.stack-guardian-serve --nodes FILE [--ctx FILE] [--port 8788]serves a fresh snapshot atGET /snapshot(read-only, loopback unless you pass--host H --allow-remote), so one machine can ask another for its health. no auth: bind to localhost or a tailnet IP only.The same server answers
GET /metricsin Prometheus text format, running the shipped rules on a fresh snapshot:stack_guardian_finding{code,severity,node}(1 while raised, 0 once resolved),stack_guardian_worst_severity(0 OK, 1 INFO, 2 WARN, 3 FAIL) andstack_guardian_last_run_timestamp.curl http://127.0.0.1:8788/metricsstack-guardian-mcp --nodes FILE [--ctx FILE] [--state FILE] [--ttl SECONDS]: see For coding agents.
Writing a rule
from stack_guardian import finding as F
def ollama_up(snapshot, ctx):
if not snapshot["nodes"]["gpu-box"].get("reachable"):
return F.make_finding("ollama.unreachable", "missing", "FAIL", "gpu-box", "Ollama does not answer")
return None
CHECKS = [("ollama.up", ollama_up)]Drop the file in a folder and point --checks-dir at it. A full example with a fix is in
examples/checks.d/20_ollama_fix.py. A rule may also return a list of findings,
one per node.
Examples
file | what it is |
| three nodes: a local one with GPUs, two remote ones over the tailnet |
| rule settings: which model gpu-box-2 should hold |
| an offline snapshot on which every shipped rule fires once, for trying |
| the lanes |
| a custom rule whose finding carries a fix |
Test
pip install pytest
python -m pytest -qCI runs this on Ubuntu, Windows and macOS with Python 3.11, plus a privacy scan of every tracked file.
License
MIT
Available Tools
3 toolsexplainExplain a finding codeBRead-onlyIdempotent
What a finding code means, what to try, and what evidence would settle it.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false, destructiveHint=false, so safety is covered. The description's real value is disclosing the shape of the response — a meaning, an action to try, and the evidence that would settle the question — which is important because no output schema exists. It does not say how unknown or malformed codes are handled.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with zero filler, front-loading the most important output (what the code means) ahead of remediation and evidence. Its only structural weakness is being a fragment with no subject or verb, which costs a little interpretive effort.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read-only lookup, the essentials are nearly covered: annotations handle the safety profile and the description sketches the return content in place of a missing output schema. What is missing is any input-format guidance and any indication of usage context, leaving the definition adequate but thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single required 'code' parameter is documented nowhere. The description at least implies the input is a finding code, but supplies no format, example, source, or case-sensitivity guidance. With one bare string parameter the description should be carrying this burden and largely does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The title supplies the verb+resource ('Explain a finding code') and the description sharpens it by naming the three things returned: meaning, remediation, and disambiguating evidence. It is a noun-phrase fragment rather than a sentence, but an agent can tell exactly what the tool produces. The siblings (stack_health, stack_snapshot) are unrelated enough that explicit differentiation is unnecessary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to call this vs. inspecting stack_health or stack_snapshot, and no prerequisites or exclusions. The agent must infer that this is the follow-up lookup for a code surfaced elsewhere. No context is given about valid inputs or when the tool is useless.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stack_healthStack healthBRead-onlyIdempotent
Graded health findings (FAIL/WARN/INFO/OK) for the AI stack's nodes, with what to try.
| Name | Required | Description | Default |
|---|---|---|---|
| node | No | one node name; omit for all |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive and open-world behavior, so the safety profile is covered. The description adds output-level context (severity grading plus 'what to try'), but says nothing about cost, scope of scanning, or auth needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence with zero waste; the primary information (what the tool returns) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, fully-annotated, single-optional-param read tool, the description is nearly sufficient, especially since it sketches the return shape in the absence of an output schema. Only the remediation-hint format and any sizing/scope caveats remain unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single optional parameter with 100% schema description coverage ('one node name; omit for all'), so the schema fully documents it. The description adds no syntax, naming, or formatting detail beyond the schema baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource and output shape: graded health findings (FAIL/WARN/INFO/OK) for the AI stack's nodes, plus remediation hints. Clear what it returns, but it does not explicitly distinguish itself from siblings stack_snapshot or explain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no exclusions, and no mention of when to prefer stack_snapshot or explain instead. The diagnostic intent is only weakly implied by 'health findings'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stack_snapshotStack snapshotARead-onlyIdempotent
The raw snapshot the checks read: Ollama endpoints, loaded models, GPUs, tailnet.
| Name | Required | Description | Default |
|---|---|---|---|
| node | No | one node name; omit for all |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=true, so the safety and mutation profile is fully covered. The description adds useful content scope (what the snapshot contains) but says nothing about response size, cost, or freshness beyond what the annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact clause with no filler, and the content list is front-loaded immediately after the resource noun. It is slightly telegraphic (a noun phrase without a verb), which costs a point on readability but not on efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so adequately by enumerating the snapshot's contents. For a zero-required-parameter, read-only tool with full schema coverage and rich annotations, this is close to complete; only return format/size details are missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the single optional `node` parameter is documented in-schema as 'one node name; omit for all.' The description adds no additional meaning about the node parameter, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies the resource precisely — a raw stack snapshot covering Ollama endpoints, loaded models, GPUs, and tailnet — so an agent knows exactly what data surface this tool exposes. The phrase 'the checks read' implicitly distinguishes it from the stack_health sibling (raw data vs. derived checks), though it never names that sibling outright.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: by framing itself as 'the raw snapshot the checks read,' it hints that an agent should call this when it wants underlying data rather than a health verdict. There is no explicit when-to-use, when-not-to-use, or named alternative (stack_health, explain) to route between.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.2.1- First observed
explain - First observed
stack_health - First observed
stack_snapshot
TDQS
Scored across 3 tools
stack_health and stack_snapshot are clearly distinct (graded findings vs raw data), but stack_health and explain overlap because both mention 'what to try.' An agent could reasonably confuse them when seeking details about a specific finding, though descriptions provide some guidance.
Two tools follow a stack_<noun> pattern while explain is a bare verb, mixing noun-phrase and verb naming conventions. The names are readable and snake_case throughout, but the inconsistent pattern prevents a higher score.
Three tools is on the low end but appropriate for a focused diagnostics server: one to report health, one to expose raw evidence, and one to explain codes. Each earns its place, though the surface is lean.
The core diagnostic loop is covered: health findings, raw snapshot evidence, and code explanations. Minor gaps exist around historical trends and remediation actions, but agents can work around these for a read-only guardian.
Maintenance
Related MCP Connectors
Live health and AI-readable metadata of invokera.com. Demo of an Invokera-hosted MCP server.
Network, domain and website diagnostics for AI clients via MCP.
AI Reasoning Cache & Consensus Layer with 11 MCP tools via Streamable HTTP.
Persistent memory, hybrid search and a goal graph for AI agents, over stdio or remote HTTP.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceSelf-hosted AI orchestrator that monitors and manages homelab services, exposing them as MCP tools for LLMs to drive.2MIT
- AlicenseNot gradedqualityAmaintenancePlug-and-play homelab dashboard in one container — GPU, local-AI VRAM, Docker, systemd, host health. Built-in read-only MCP server so AI agents can explore it too.219MIT
- AlicenseNot gradedqualityBmaintenanceA locally-run, read-only MCP server that lets an LLM client diagnose a self-hosted *arr media stack by aggregating across Sonarr, Radarr, Prowlarr, qBittorrent, Tdarr, and Profilarr.Apache 2.0
- FlicenseNot gradedqualityBmaintenanceAI sysadmin for Home Assistant that provides entity health analysis, automation diagnostics, dashboard analysis, configuration search, and system health scoring via the Model Context Protocol.-