Skip to main content
Glama

it-desk

A self-healing monitor and ticket desk for a fleet of local HTTP services on one Windows machine. It probes each service, restarts the ones that fail, verifies that exactly one process is serving the port afterwards, records every step as a ticket with evidence, asks a model for a diagnosis when scripted restarts stop working, and notifies a human. Coding agents reach it through an MCP server so they can check health, file bugs, request restarts and work a shared ticket queue instead of killing processes themselves.

It is a single FastAPI process (run.py) with a SQLite store, a static dashboard, and a stdio MCP adapter (mcp_server.py).

Architecture

probe (HTTP, bounded timeout)
  -> state machine per service (GREEN / RED / BREAKER, startup grace, consecutive-red gate)
  -> diagnose (process dead, orphan listener, zombie duplicates, hang)
  -> restart: drain_url (optional) -> exclusive restart lease (optional broker)
              -> kill by command-line regex (denylist enforced) -> launch from registry
              -> verify listener + probe
  -> ticket with timeline events (probe_fail, diagnosis, kill, launch, verify_ok/verify_fail, ...)
  -> recovered: ticket auto_resolved
  -> scripts give up (verify fails, breaker trips, check_only service down): escalate
       -> AI triage (structured JSON first, prose fallback; diagnosis only, never executed)
       -> notification (Discord webhook and/or signed off-site heartbeat)
  -> humans or agents claim / release / resolve tickets through the dashboard or MCP

Main modules:

Module

Role

app/monitor.py

Poll loop, per-service state machine, restart policy, breaker, escalation

app/probes.py

HTTP health probes

app/diagnose.py

Tier-1 classification before anything is touched

app/remediate.py

The only action surface: drain, lease, kill-by-match, launch, verify

app/procs.py

Process and listener helpers; the never-kill denylist lives here in code

app/tickets.py

SQLite (WAL) store: tickets, events, probe history, restart log, playbooks

app/triage.py

Tier-2 model diagnosis; output is attached to the ticket only

app/actions.py

Catalog of verbs a model may select; model-selected actions are recorded, only the monitor executes restarts

app/storm.py

Groups 3+ services failing within 120 s into one fleet event

app/heartbeat.py

Optional signed heartbeat to an off-site watchdog (dead-man alerting)

app/explain.py

Optional plain-language explanation feed over open tickets

app/prime.py, app/prime_routes.py

Optional persistent Claude Code session in a ConPTY, shared over a WebSocket

app/scrub.py

Secret scrubber applied to every off-machine or untrusted-text sink

app/routes.py

Dashboard and JSON API

mcp_server.py

stdio MCP server proxying to the desk

More detail: docs/ARCHITECTURE.md.

The state-0 listener problem

Restart verification asks "is exactly one live process serving this port?" (procs.verify_listener). On some Windows builds a healthy uvicorn listener is reported with TCP state 0 (psutil status NONE): netstat prints no LISTENING row and Get-NetTCPConnection shows a blank state. A filter on LISTEN therefore sees no owner for a working service, every restart is recorded as failed, and the failures trip the circuit breaker.

procs.port_owners counts any socket bound to a loopback or wildcard address on the port with no remote end and an owning PID that is alive, whatever state it reports. Connected sockets (ESTABLISHED, TIME_WAIT) and binds on other addresses (for example a VPN proxy re-listening on the same port) are ignored. verify_listener then returns:

  • pid-bound: exactly one owner and the probe passed;

  • http: no socket row visible but the probe passed;

  • failure when the probe failed or two or more processes own the port (zombie duplicates).

The netstat fallback (used only when psutil cannot read the table) is still blind to state-0 rows. Tests: tests/test_listener_probe.py.

Related MCP server: DevContext

MCP tools

Register: claude mcp add -s user it-department -- py -3.12 <path-to-checkout>/mcp_server.py

Tool

Purpose

it_status

Fleet health plus open ticket counts by kind

it_report

File a bug the agent could not fix (merged with an open ticket of the same normalised title, reopened if closed within 7 days)

it_restart

Request a coordinated restart (drain, lease, kill, relaunch, verify, ticketed)

it_investigate

Open an investigation ticket with evidence

it_queue

List tickets ready to work

it_claim / it_release

Take or hand back a ticket

it_resolve

Close a claimed ticket with a resolution

Writes carry a session id (CLAUDE_CODE_SESSION_ID or a random one). Inside the prime terminal (IT_PRIME=1) writes are stamped as the prime, and the desk refuses restarts from that source.

Running

Requirements: Windows, Python 3.12.

py -3.12 -m pip install --require-hashes -r requirements.lock
copy config\services.example.yaml config\services.yaml   (then edit)
mkdir %USERPROFILE%\.it-desk
py -3.12 -c "import secrets; print(secrets.token_urlsafe(32))" > %USERPROFILE%\.it-desk\token
py -3.12 run.py

The dashboard is at http://127.0.0.1:8107. The API and dashboard accept loopback Host headers only, and every write needs Authorization: Bearer <token>. With no token configured, writes fail closed.

Configuration

Environment

Variable

Default

Meaning

IT_DESK_SERVICES

config/services.yaml, else config/services.example.yaml

Service registry path

IT_DESK_SETTINGS

config/settings.yaml

Settings path

IT_DESK_DATA_DIR

data/

SQLite DB (itdept.db), caches, prime state

IT_DESK_LOG_DIR

logs/

itdept.log (self-trimming)

IT_DESK_TOKEN_FILE

~/.it-desk/token

Bearer token file (read by the desk and the MCP server)

IT_DESK_TOKEN

unset

Token fallback when the file is absent

IT_DESK_URL

http://127.0.0.1:8107

Desk URL used by mcp_server.py and scripts/migrate_bugs.py

IT_DESK_LEASE_BROKER_URL

http://127.0.0.1:8106

Optional lease broker for exclusive restarts; unreachable = leaseless

IT_DESK_BEAT_SECRET_FILE

~/.it-desk/beat_secret

DPAPI-encrypted HMAC key for the heartbeat

IT_DESK_BUGS_DIR

~/.it-desk/bugs

Markdown mirror of bug tickets (BUGS.md) and the offline bugs.jsonl inbox

IT_DESK_REAPER_LOG

unset

Optional log of an external orphan-MCP reaper; feeds the MCP-zero alert and /api/logs/reaper. Unset = off

IT_DESK_CLAUDE_PROJECTS

~/.claude/projects

Where Claude Code writes session transcripts; the prime terminal reads its own transcript there for context usage

IT_DESK_AUDIT_LOG

unset

Optional JSONL audit log for the consultation-storm detector; unset = off

IT_DESK_PROVIDER_CHAIN

unset

Optional module:function called as fn(prompt, system) returning {"text", "provider", "errors"}; tried before the built-in providers

DEEPSEEK_API_KEY

unset

Key for the built-in DeepSeek provider

CONDUCTOR_URL

unset

Optional agent-run conductor whose approval gates appear in the explain feed

IT_DESK_DB

unset

DB path for scripts/stats.py

IT_PRIME

unset

Set to 1 by the prime terminal for its MCP child

config/services.yaml

A list under services:. See config/services.example.yaml. Validation is in app/config.py:_validate_services and runs on every save through the API.

Key

Meaning

id

Unique id

name

Display name

type

managed (may be restarted) or check_only (alert only)

port

Port verified after a restart. Omit on check_only rows

probe.url

Health URL; must start with http://127.0.0.1

probe.expect_status, probe.timeout_s

Expected HTTP status and timeout

launch.command

argv list; [0] must be py, python.exe, or an absolute path ending in python.exe (bare python is refused)

launch.cwd

Absolute working directory

launch.env

Optional UPPER_SNAKE: "string" map layered over the desk's environment

kill_match

Regex over the process command line; at least 6 characters, must contain a backslash or .py, no catch-alls

drain_url

Optional; POSTed before a kill so the service can finish in-flight work (15 s deadline)

startup_grace_s

Probe failures within this window after a launch are ignored

depends_on

Parent id; a child failing while its parent is down is grouped under the parent

enabled

false skips the service

config/settings.yaml

Key

Default

Meaning

poll_interval_s

15

Probe interval (minimum 3)

fail_threshold

2

Consecutive probe failures before a service goes RED

restart_verify_wait_s

8

Wait before verifying a relaunch

probe_history_days

7

Probe history retention

stale_incident_min

30

An incident open this long, given up on by the monitor, appears in the work queue

bug_autoclose_green_min

30

A bug linked to a service closes after the service has been green this long

circuit_breaker.max_restarts_per_hour

4

Restart budget per service; exceeding it opens the breaker and escalates

circuit_breaker.cooldown_min

30

Breaker open time

restart_policy.min_gap_s / max_gap_s

60 / 900

Backoff between restarts of one service

restart_policy.reset_after_green_s

600

Green time that resets the backoff

restart_policy.consecutive_red

{}

Per-service RED ticks required before the first restart

triage.provider

claude_cli

claude_cli, deepseek or off; the other built-in provider is the fallback. DeepSeek is only usable when DEEPSEEK_API_KEY is set; otherwise it fails fast with that message

triage.claude_cmd

["claude", "-p", "--tools", "", "--strict-mcp-config"]

Command for the claude_cli provider (prompt on stdin). The default disables all built-in tools and all MCP servers, so injected ticket text cannot make the CLI act. Keep those flags if you override it

triage.timeout_s

180

Provider timeout

triage.max_runs_per_ticket

1

Triage runs per ticket

discord.webhook_url

''

Notification webhook; blank = no-op

discord.notify

[escalated, recovered_after_escalation]

Events that notify

dashboard.host / port

127.0.0.1 / 8107

Bind address; host must stay loopback

heartbeat.enabled, url, interval_s, timeout_s, sleep_hours, critical_services

off

Signed off-site heartbeat; see app/heartbeat.py

paused

false

Monitor loop paused (no probes, no restarts)

consult_storm.enabled, window_min, threshold, every_s

on, 10, 5, 300

Consultation-storm detector (needs IT_DESK_AUDIT_LOG)

cli_version_check.enabled, every_s

on, 3600

Notes Claude Code CLI version changes on matching tickets

Incident statistics

py -3.12 scripts/stats.py [--db PATH] [--days N]

Prints incident tickets opened in the window, recovered without escalation, escalated, went green after escalation, restart budget exhausted, closed with a written resolution, still open, median and p90 time from open to recovery without escalation, and a per-service breakdown. Bugs and investigations filed by agents are not counted. The database is opened read-only.

Measured figures

Figures measured on the author's own fleet over 114 days (2026-07-05 to 2026-10-26) with scripts/stats.py; the database is not published.

Measure

Value

Incidents opened

1543

Recovered without escalation (green again before any escalation; includes restarts and transient blips)

1123 (72.78%)

Escalated (an escalation_reason note exists)

407 (26.38%)

Escalated, then went green again (automatic or manual)

405 (26.25%)

Restart budget exhausted (breaker opened)

4 (0.26%)

Closed with a written resolution by a person or agent

11 (0.71%)

Open to recovered without escalation, median / p90

16 s / 58 s

Most escalations come from check_only services, which are never restarted and escalate on first failure; one such service accounts for 338 of the 407.

Threat model

The desk assumes a single trusted local user.

  • Read routes (GET /api/status, /api/tickets, /api/bugs, /api/settings and similar) are unauthenticated local reads. They expose ticket text, error output and settings (the Discord webhook URL is redacted). Anything that can reach 127.0.0.1:8107 with a loopback Host header can read them. GET /api/registry and the prime routes require the token.

  • A bearer token holder can change config/services.yaml (launch.command, launch.cwd) and triage.claude_cmd through the API. That is authenticated local code execution, by design: the desk exists to launch processes.

  • The token is a file in the user profile (IT_DESK_TOKEN_FILE). Any process running as that user can read it.

  • The prime-source check (X-IT-Source: prime header or source/who body fields) is a convenience guard that stops the prime session from requesting restarts. It is not a security boundary; any token holder can omit it.

  • Ticket text is treated as untrusted: it is fenced and scrubbed before it reaches a model, and the default triage CLI runs with no tools.

Tests

py -3.12 -m pip install -r requirements-dev.txt
py -3.12 -m pytest -q

tests/conftest.py redirects every data, log and config path to a temporary directory and fails the session if anything writes under the repo's data/, logs/ or config/. Cross-checks that need the optional winspawn spawn auditor are skipped when it is not installed. CI runs on windows-latest.

Limitations

  • Windows only: process handling, ConPTY (pywinpty), DPAPI secret storage, CREATE_NO_WINDOW spawns and the listener semantics above are Windows-specific. requirements.lock pins pywinpty, so it does not install on Linux.

  • Managed services must be Python processes launched from the registry; kill_match selects processes by command line.

  • One machine. Probes must target 127.0.0.1; the dashboard binds loopback only.

  • AI triage is advisory. Model-selected actions are recorded; only the monitor executes restarts.

  • The prime terminal needs the Claude Code CLI on PATH, ~/.it-desk/prime_mcp.json and ~/.it-desk/prime_settings.json.

  • The bug mirror writes BUGS.md into IT_DESK_BUGS_DIR on every bug write.

License

MIT. See LICENSE.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    A self-hosted work dashboard for AI agent teams, where agents create and manage tickets through MCP tools and humans review an awaiting-human queue backed by an append-only event log. Enables ticket lifecycle management and evidence-based provenance tracking.
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables coding agents to access live runtime observability data such as logs, deploys, and health metrics for evidence-based incident triage.
    1
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables multiple AI coding agents to coordinate on a shared software project by registering, claiming tasks, declaring file intents, publishing structured change reports, and handing off context, with a local dashboard showing state in near real time.
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    Local OS for your AI Agents fleets. ——————- Enables AI agents to coordinate through a durable local board with shared state, ticket lifecycle, evidence-based approvals, and journal-woken handoffs.
    2
    Apache 2.0