# SysMCP

An MCP server that exposes real Linux production-debugging primitives — `perf`,
`strace`, `/proc`, flamegraphs, eBPF — as safe, structured, agent-callable
tools, so an agent can autonomously diagnose *why a service is slow* instead of
reading logs and guessing.

```
      agent
        │
        ▼
   ┌─────────────────────────────────────────────┐
   │  SysMCP                                     │
   │  ┌────────────┐  ┌──────────┐  ┌─────────┐  │
   │  │ safety     │  │ session  │  │ probe   │  │
   │  │ allowlist  │  │ artifact │  │ what    │  │
   │  │ path jail  │  │ handles  │  │ works   │  │
   │  │ budgets    │  │ + cache  │  │ + how   │  │
   │  │ redaction  │  │          │  │ to fix  │  │
   │  │ audit      │  │          │  │ it      │  │
   │  └────────────┘  └──────────┘  └─────────┘  │
   │  ┌───────────────────────────────────────┐  │
   │  │ 11 tools → ranked, budgeted evidence  │  │
   │  └───────────────────────────────────────┘  │
   │  ┌───────────────────────────────────────┐  │
   │  │ Executor: local │ wsl │ ssh │ docker  │  │
   │  │            │ replay (record/replay)   │  │
   │  └───────────────────────────────────────┘  │
   └─────────────────────────────────────────────┘
                        │
                        ▼
              the machine being debugged
       perf · strace · bpftrace · /proc · /sys
```

## Status, stated plainly

The server, its safety layer, 11 tools, five transports, 15 benchmark scenarios
and the full benchmark harness are built and tested — **238 tests, all
passing**, with a mutation check confirming 12 of 12 known defects are caught
by an existing test.

**No benchmark result exists.** This was built on a Windows host with no Linux
target available, so the debugging tools have never run against a real kernel.
The claim the project was designed to support — *"agents with SysMCP identify
root causes X% more often"* — is **not supported by anything here**. What has
and has not been validated is set out precisely in
[docs/04-benchmark.md](docs/04-benchmark.md#what-has-and-has-not-been-validated),
and the build log with every bug found along the way is in
[docs/JOURNAL.md](docs/JOURNAL.md).

Getting a real result needs one `probe` and one `record` run against a Linux
host: see [docs/03-targets.md](docs/03-targets.md#getting-a-real-target).

## Why this is not a CLI wrapper

Nearly every public MCP server is a CRUD wrapper. Wrapping `perf` the same way
— one `run_command(cmd)` tool — fails for three reasons, and the whole design
follows from avoiding them.

**1. Output volume.** `strace` on a busy process emits megabytes per second;
`perf script` on a 5-second profile emits tens of megabytes. An agent handed
that verbatim has no context left to reason with. So every tool **ranks, then
truncates**, and says when it truncated. Ranking before truncating is what makes
the surviving data the *useful* part.

**2. Heavyweight intermediate results.** `perf record` produces a binary
`perf.data`. Moving it through a language model is absurd; re-recording for
every follow-up question is worse, because each recording samples a *different
moment* and the answers no longer compose. So SysMCP has an **artifact model**:
heavy output stays on the target, the tool returns a handle plus a small
summary, and later tools query the handle. One recording answers an entire
investigation for a few hundred tokens.

**3. Choosing the wrong instrument.** A CPU profiler is blind to time a process
spends *not running*. An agent that reaches for `perf record` on a process
blocked in `fsync` sees an idle machine and concludes the service is healthy.
So `process_inspect` returns the numbers that **classify** the bottleneck
before any profiler runs, and every tool description says *when to use it*.

There is deliberately **no generic shell tool**, and there never will be — it
would collapse every safety layer at once. See
[ADR 0002](docs/adr/0002-no-generic-shell-tool.md).

## Quick start

```bash
uv venv .venv && uv pip install --python .venv -e ".[dev]"

# What can SysMCP actually do on this target, and how do I fix what it can't?
python -m sysmcp --target ssh:deploy@10.0.3.7 probe

# Try a tool directly and see the JSON an agent would get
python -m sysmcp --target wsl:Ubuntu-24.04 call process_list \
    --params '{"interval_s": 1, "top_n": 10}'

# A whole investigation in one session, offline, no Linux needed
python scripts/walkthrough.py --target replay:fixtures/linux
```

Register with an MCP client:

```json
{
  "mcpServers": {
    "sysmcp": {
      "command": "python",
      "args": ["-m", "sysmcp", "--target", "ssh:deploy@10.0.3.7", "serve"],
      "env": { "SYSMCP_AUDIT_LOG": "/var/log/sysmcp/audit.jsonl" }
    }
  }
}
```

## The tools

Eleven, not thirty. Each answers a question an investigation actually asks.

| Tool | Answers | Cost |
|---|---|---|
| `capabilities` | what works here, and the exact fix for what doesn't | ~3 s, cached |
| `host_snapshot` | is the **host** saturated (CPU / iowait / swap / run-queue / PSI)? | one interval |
| `process_list` | which process — with CPU **measured**, not `ps`'s lifetime average | one interval |
| `process_inspect` | classify one process: user vs system CPU, memory, real vs cached I/O, context switches, per-thread state, fds vs `NOFILE` | one interval |
| `perf_record` | sample stacks (`cpu`/`off-cpu`/`page-faults`/`context-switches`/`cache-misses`) → artifact; or `analysis="counters"` for IPC and rates | `seconds` |
| `perf_report` | ranked symbols by **self** or **total** time; `mode="callers"` for who calls what | free after first query |
| `flamegraph` | call-tree structure, plus a self-contained SVG for a human | free |
| `syscall_trace` | ranked syscall time; `mode="detailed"` groups calls **by subject** | expensive — capped |
| `memory_profile` | growth rate, whether it's **monotonic**, composition, PSS vs RSS | one interval |
| `inspect_source` | bounded, line-numbered source at a symbol | trivial |
| `flamegraph_from_folded` | analyse stacks from py-spy, async-profiler, or an old incident | free |

Details, output shapes and the reasoning behind each:
[docs/02-tools.md](docs/02-tools.md).

Three details that carry disproportionate weight:

- **`process_list` measures CPU.** `ps`'s `%CPU` is a lifetime average: a
  worker up for three weeks and pinned for the last five minutes reads ~0.2%.
  An agent handed that concludes the process is idle, and no later profiling
  recovers from starting in the wrong place.
- **`syscall_trace` groups by subject.** "`openat` called 12,041 times" is a
  clue; "`openat` called 12,041 times on `/etc/app/config.yaml`" is the
  diagnosis. Grouping is exactly what an agent cannot do well by reading raw
  trace text.
- **`perf_report(mode="callers")` is where root causes appear.** "82% of CPU in
  `parse_config`" is an observation. "`parse_config` is reached from
  `handle_request` on 100% of samples" is the cause.

## Safety

The agent is treated as *semi-trusted*: not malicious, but steerable by the very
data it reads — process names, command lines and log text are all
attacker-controllable on a compromised host and all flow back into the model.
So refusals are structural, not requests.

1. **No string interpolation into commands, ever.** Tools take typed parameters
   and build argv vectors programmatically.
2. **Leaf validation** on every agent-supplied value.
3. **argv gate** against a binary allowlist, with wrapper unwrapping so
   `timeout 5 <anything>` cannot slip past.
4. **Read-only by construction.** Every allowlisted binary is an observer; no
   tool can signal a process, write to `/proc/sys`, or drop caches.
5. **Bounded cost** — timeouts, output caps, sampling ceilings, a concurrency
   gate that fails fast rather than queueing.
6. **Redaction.** `/proc/<pid>/environ` is never returned in full; command
   lines and argv are scrubbed in both the transcript and the audit log.
7. **Append-only audit log** that replays as a shell script, so a human can take
   over an investigation an agent started.

Full threat model, including what these controls do **not** cover:
[docs/01-safety.md](docs/01-safety.md).

## Repository layout

```
src/sysmcp/
  server.py          MCP tool registration; descriptions are prompt engineering
  session.py         artifact handles, caches, budgets
  probe.py           functional capability detection with actionable remedies
  safety/            allowlist + path jail, limits, redaction, audit
  targets/           local, WSL, SSH, Docker, record/replay
  parsers/           /proc, perf report+stat, strace, folded stacks, SVG
  tools/             the 11 tool implementations
services/            one service, 16 selectable faults, + verifier + deployer
scenarios/           15 benchmark scenarios, generated from the fault registry
bench/               runner, grader, metrics, HTML dashboard
fixtures/            synthetic replay corpus (generated, clearly labelled)
tests/               238 tests
scripts/             end-to-end walkthrough
docs/                design, safety, tools, targets, benchmark, ADRs, journal
```

## Development

```bash
.venv/Scripts/python -m pytest -q                  # 238 tests, no Linux needed
.venv/Scripts/python -m ruff check src tests bench services scenarios
.venv/Scripts/python scripts/walkthrough.py        # end-to-end, offline
.venv/Scripts/python services/verify_faults.py     # do the faults still manifest?
.venv/Scripts/python services/deploy.py --bug none --all --check  # do artifacts leak answers?
.venv/Scripts/python scripts/mutation_check.py     # are the tests vacuous?
.venv/Scripts/python scripts/stats.py              # authoritative counts
```

The test suite runs against a replay fixture corpus, so it needs no Linux, no
privileges and no network, and is deterministic. What that cannot cover is the
interaction with real `perf`/`strace` binaries — closed by `sysmcp record`
against a real host.

## Documentation

| | |
|---|---|
| [docs/00-design.md](docs/00-design.md) | architecture and the decisions behind it |
| [docs/01-safety.md](docs/01-safety.md) | threat model, controls, and their gaps |
| [docs/02-tools.md](docs/02-tools.md) | tool reference and output shapes |
| [docs/03-targets.md](docs/03-targets.md) | transports, setup, getting a real target |
| [docs/04-benchmark.md](docs/04-benchmark.md) | the experiment, and what it does and does not show |
| [docs/05-scenarios.md](docs/05-scenarios.md) | the 15 fault scenarios |
| [docs/adr/](docs/adr/) | decision records |
| [docs/JOURNAL.md](docs/JOURNAL.md) | build log: every bug found, and how |