slurm-mcp

A read-only MCP server exposing Slurm scheduler state to agents. The allowlist is enforced in code, not requested in a prompt, and the tool surface uses progressive disclosure so the flag surface of seven Slurm binaries is not resident in the agent's context on every turn.

v0.2.0 (unreleased). The guard, the topic surface and the stdio server are built. The tests drive the guard adversarially, run the live path against fake Slurm binaries, and exercise the MCP transport in-process and over real stdio against fixtures. It has not been run against a real Slurm cluster, and the fixtures are hand-written, not recorded. See CHANGELOG.md for what changed since v0.1.0 and why.

Why the guard is in code

A system prompt saying "only use read-only commands" is a request, not a control. It fails open. A jailbreak, a confused tool call, or an ordinary hallucination is enough to reach scontrol update on a production controller.

Anything that could drain a node must be impossible to express, not merely discouraged. So the allowlist lives in guard.py and runs on every invocation. It is an allowlist of shapes, in two layers that must both pass:

• Binary shape. Only seven read binaries may run, and each may carry only the exact option spellings listed for it. No abbreviations, no bundled short options, no option in front of a subcommand. scontrol may run only as scontrol show <config|job|node|partition> [name]. sdiag takes no options, because sdiag --reset zeroes the counters a diagnosis reads.

• Topic shape. The argv must be exactly one topic's command followed by that topic's own filter flags. Every filter value must be a plain name, list, state or time: it cannot start with - and cannot contain whitespace, a shell metacharacter or a control character.

• Commands execute with shell=False anyway. That is defence in depth, not the only barrier.

v0.1.0 did this with a list of forbidden subcommands, and an audit found 23 argv that passed it and should not have. Twenty-two come from Slurm's own behaviour, and each writes, mints a credential, resets counters or opens an interactive prompt. Slurm accepts unique abbreviations (scontrol upd is update). An option with a value can hide the verb (scontrol -u 0 shutdown). Verbs were missing from the list (uhold, token, sacctmgr shutdown). A bare scontrol reads commands from stdin. The 23rd was v0.1.0's own: a filter value starting with --format= skipped the metacharacter check. All 23 are in tests/test_guard.py, and each is refused by either layer on its own. Each of the 22 upstream cases cites the man page or source line it relies on.

The threat model is not a malicious user. It is an agent that has read a confusing log line at 3am and is about to do something decisive.

The tests drive the guard with 70 adversarial argv (a test checks that this number matches the corpus) rather than asserting on prompt text, because a prompt-level promise cannot be tested:

make test

Related MCP server: mcp-slurm

Deployment: Slurm's own authorization is the backstop

The guard decides what this server will ask Slurm to do. Slurm decides what the asking account is allowed to do, and that is the stronger control. Run the server as a dedicated, unprivileged Slurm user: no AdminLevel, not an operator or coordinator. Then a guard bug that let a write through would still be refused by slurmctld. (sdiag --reset, for example, is "Only supported for Slurm operators and administrators", per sdiag(1).)

Two consequences of that choice:

• accounting runs sacct -a and fairshare runs sshare -a. Per sacct(1), -a shows all users' jobs only "when run by user root or if PrivateData is not configured to jobs"; otherwise it shows the server account's own jobs. Depending on your PrivateData, the agent sees either everyone's jobs or almost nothing. Decide which you want before deploying.

• While serving, every live read is logged to stderr as one bare JSON object per line, with argv, exit_code, seconds and cached. Reads answered from the cache are logged too, with "cached": true, although no Slurm command ran for them. Calls refused by the guard or the gate run nothing and are not logged. stdio servers may write to stderr freely, and an SRE-facing server should leave its own audit trail rather than rely on the client's.

Load on slurmctld

A read-only server can still hurt a struggling controller. The squeue man page's PERFORMANCE section says: "Do not run squeue or other Slurm client commands that send remote procedure calls to slurmctld from loops in shell scripts or other programs." An agent retrying slurm_overview is exactly that loop. The MCP spec also says servers MUST "Rate limit tool invocations" (tools, 2026-07-28).

So every live read goes through a gate:

Concurrent identical calls share one subprocess while the cache is on (--cache-ttl above 0). With it off they take turns, and each runs its own. A cached answer is labelled [live, cached 4s ago] and does not spend the rate budget. A refused call says why: the rate-limit refusal says when to retry, and a busy refusal says every slot stayed full for 20 s. Output past the row cap ends with [truncated: 200 of N rows shown; narrow with filters: …], so a Slurm-sized answer does not become a Slurm-sized context window. Slurm calls run in a worker thread, so the server keeps answering other requests while slurmctld is slow.

The defaults are judgement calls, not tuned against a real controller. They apply per server process: ten clients each launching their own stdio server get ten times the budget.

Progressive disclosure

The obvious design exposes one tool per binary — sinfo, squeue, sacct, sdiag, sprio, sshare, scontrol — each carrying a schema for its flags. Slurm's flag surface is enormous and most of it is irrelevant to any given question, but all of it sits in context on every turn.

This server exposes three tools, all annotated readOnlyHint: true:

Measured with make footprint (output of slurm-mcp footprint at v0.2.0; a test fails if these numbers drift from the code):

  resident: three tool descriptors    1486 chars
  resident: server instructions        369 chars
  resident total                      1855 chars
  detail, fetched on request          4229 chars
  flat, 7 schemas only                2009 chars  (1.08x resident)
  flat, schemas + inlined detail      6238 chars  (3.36x resident)

Read this carefully, because the headline depends on an assumption. A flat server with one generic tool per binary and no guidance is about the same size as this server's resident surface (1.08x). The 3.36x only holds if a flat server would carry guidance comparable to slurm_describe in its tool descriptions. That assumption is not measured against any real server. The resident side counts the server instructions, sent at initialize; the flat side is given none, which favours the flat design. What the measurement does show is where the 4229 characters of detail live: here, off the resident path until asked for.

That is a context-cost measurement, not a quality claim. It does not say the agent will diagnose anything better. Nothing here has been scored on slurm-rca-bench.

v0.1.0 reported 1088 / 3029 / 4441 chars (4.1x). Those numbers reproduced, but the ratio came mostly from the unstated inlined-detail assumption, the flat baseline included sacctmgr (which no topic used), and the instructions were not counted.

Topics: queue, nodes, accounting, priority, fairshare, diagnostics, config.

Quickstart

No cluster required. make install uses uv when it is on PATH, and otherwise needs PY to be Python 3.11 or newer.

make install
make surface      # the read-only allowlist, and what is refused
make demo         # a full overview against the hand-written fixtures
make footprint    # reproduce the context measurement above
make check        # ruff, ruff format, mypy --strict, pytest
make smoke        # the MCP server over real stdio, against fixtures

Against a real cluster, drop --fixtures:

slurm-mcp overview
slurm-mcp query queue --filter user=alice
slurm-mcp describe accounting

overview and query exit non-zero when any read failed, so a script can tell "the read failed" from "nothing is queued".

As an MCP server

From a checkout:

pip install -e ".[server]"      # or: uv pip install -e ".[server]"
slurm-mcp --fixtures serve      # stdio; drop --fixtures to read the cluster

Or straight from git. This is the standard pip form, but this exact command has not been run while the branch is unpublished. The equivalent non-editable install of a local checkout has been, and the CI workflow repeats it from a built wheel.

pip install "slurm-readonly-mcp[server] @ git+https://github.com/Zhanyl-tech/slurm-mcp"

The distribution is slurm-readonly-mcp because slurm-mcp on PyPI is an unrelated project; pip install slurm-mcp installs someone else's code. The command is still slurm-mcp. The server needs mcp>=2.0,<3: it is written against the mcp 2.x API, and every mcp 1.x release fails at start-up.

Then point an MCP client at the command. Clients need the absolute path. For Claude Code, in a project .mcp.json:

{
  "mcpServers": {
    "slurm": {
      "command": "/absolute/path/to/slurm-mcp/.venv/bin/slurm-mcp",
      "args": ["--fixtures", "serve"]
    }
  }
}

or claude mcp add --transport stdio slurm -- /absolute/path/to/.venv/bin/slurm-mcp --fixtures serve (format). Every response is labelled [live] or [fixture] so sample data can never be mistaken for a cluster read. In fixture mode the user, partition and account filters are applied by the server to the sample rows, and the response says so, truncated or not. Other filters are refused rather than imitated, and a fixture-mode truncation hint does not suggest them.

Why the CLI, and when slurmrestd would be better

This server runs the Slurm client commands. That needs the client binaries, a slurm.conf and working Slurm authentication (typically munge) on the host, which suits a login node or an admin box. It also means parsing text with the column legends in topics.py.

Slurm also "provides a REST API through the slurmrestd daemon, using JSON Web Tokens for authentication" (rest.html, Slurm 26.05). Slinky's slurm-operator "obtains a JWT so it can talk to each Slurm cluster it manages via slurmrestd" (architecture). For a server running as a pod next to a Slinky-managed cluster, a slurmrestd backend would avoid client binaries and munge in the pod and would return typed JSON. Its read-only property would then rest on two things: a closed list of GET requests, and a JWT for a Slurm user with no admin or operator rights, so slurmctld refuses writes even if the server had a bug. It is also why the guard refuses scontrol token: minting a JWT is not a read.

That backend does not exist here. Neither does --json parsing into MCP structuredContent, although several of these commands offer --json. Both are deferred; the tool surface would stay the same three tools.

Prior art

Other public Slurm MCP servers exist. The ones checked (their READMEs, read 2026-09-26) expose write paths:

• dongwookim-ml/slurm-mcp — submit_job, cancel_job, plus file management and shell access.

• yidong72/slurm_mcp — 34 tools over SSH, including submit_job, cancel_job, hold_job.

• sahuno/slurm-mcp — submit, monitor and cancel jobs, including --wrap commands.

• tejasvinu/mcp-slurm — submit, and cancel / hold / release / suspend / resume / requeue / modify jobs.

This one differs in scope, not quality. It is read-only by construction, with the allowlist enforced in code and tested adversarially; the agent picks from a closed vocabulary of topics rather than writing a command line; and detail is disclosed progressively.

Limitations

• Read-only is enforced for this server's own surface. It does not sandbox the host, and it says nothing about what other tools an agent has been given. Slurm authorization is the backstop (see Deployment).

• The fixtures are hand-written and illustrative, not recordings. They are shaped like each command's output so the surface is runnable; no result should be quoted from them. See fixtures/README.md.

• Column layouts follow the Slurm 26.05 man pages (checked 2026-09-26), and are unverified against a live cluster. Older or newer Slurm versions may print differently. Verifying this needs a real or containerised Slurm, and fixtures recorded from it with the version and command in a header.

• No scoring. The claim here is about the tool surface, not diagnostic accuracy. Measuring whether progressive disclosure changes an agent's diagnosis is cluster-sre-agent's job, and it has not been done.

• Limits are per process, and output is capped after capture. The row cap bounds what reaches the agent, not what the subprocess writes into this server's memory first.

• sacct can block. A degraded accounting path makes it hang rather than error; calls time out at 20s and say so, because an agent waiting forever is worse than an agent told the read failed. (In slurm-rca-bench scenario S01 — one run, Slurm 25.11.4 in Docker Compose — sacct blocked with no error while scheduling continued.)

Related

• cluster-sre-agent — the agent that consumes a surface like this one. Its internal read-only tool layer is where this design came from; this repo is the standalone server. It keeps its own read-only guard. The two are separate code, not a shared module, so a fix to one does not reach the other.

• cluster-ops-skills — the runbooks an agent follows once it can read the cluster.

• slurm-rca-bench — where a claim about diagnostic quality would have to be proven.

License

MIT.