# dtu-hpc

**Let your coding agent run your DTU HPC jobs — without ever giving it a shell.**

`dtu-hpc` connects MCP-capable agents (Claude Code, Codex, and others) to the
DTU Computing Center cluster through a small, audited set of operations. You
authenticate once a day; your agent submits, monitors, chains, and diagnoses
LSF jobs while you direct the research. No SSH terminal, no Cisco VPN in your
daily loop, and no interactive shell on the login node — every action the agent
can take is a named, argument-validated operation under limits it cannot
raise.

## Why not just give the agent SSH?

Because your student account is personally accountable for everything that
happens on a shared, rationed cluster — and agents *are already being used
over raw SSH today*, with no guardrails at all. This tool is the safer path
for something that is happening anyway:

- **No shell on the login node.** Agents choose from named operations (submit
  job, check queue, fetch logs). Every argument is validated. There is no "run
  this command", and agents never supply script text — they send a spec, and
  the trusted daemon writes the job script itself.
- **Jailed writes.** Agents can only touch approved project folders, with paths
  re-checked after the server resolves them, so a planted symlink can't escape.
  Your dotfiles and SSH keys are structurally out of reach.
- **Hard limits the agent cannot raise.** GPUs, cores, memory, walltime, chain
  length, concurrent jobs, and submissions per hour, enforced inside the
  credentialed daemon from a file outside your project. Raising one takes a
  human edit plus a restart.
- **Audit trail.** Every action — including every denied one — is logged
  locally. You can always answer "what did my agent do on the cluster?"
- **One kill switch.** All access flows through one daemon on your laptop.
  Stop it and every agent loses cluster access instantly. (Jobs already queued
  keep running on DTU — `dtu-hpc down` tells you which, so you can cancel them
  first if that's what you meant.)
- **Nothing installed on the cluster.** DCC sees ordinary SSH sessions and
  ordinary, well-formed job submissions. No daemons on login nodes.
- **No telemetry.** This tool never phones home. Your usage data is yours.

## What your agent gets good at

DCC runs **LSF**, which most tutorials (and most LLMs) get wrong — they emit
SLURM syntax the cluster rejects, or omit the flags that silently kill jobs at
15 minutes and 1 GB of RAM. The templates here encode the cluster's real rules,
including the one that matters most for ML work:

**GPU jobs die at 24 hours.** `dtu-hpc` treats longer runs as first-class:
`hpc_submit_chain` splits a 72-hour training run into three checkpointed,
dependency-linked segments that resume from each other, with LSF signalling
your trainer ten minutes before each kill so it can flush a checkpoint.

## Architecture

```
Claude Code / Codex        agents: disposable, hold no secrets
        │  stdio (MCP)
   dtu-hpc MCP server      stateless adapter
        │  loopback + per-run token
      dtuhpcd              you type your password HERE, once — nowhere else
        │  one persistent SSH session
 login.hpc.dtu.dk → LSF    cluster sees a normal user session
```

## Setup (once)

```
# Not on PyPI yet — install from source:
git clone https://github.com/PV-Bhat/dtu-hpc-mcp && cd dtu-hpc-mcp
python -m venv .venv && .venv/Scripts/activate     # Linux/macOS: source .venv/bin/activate
pip install -e .
dtu-hpc init               # key setup + cluster install + agent config snippets
```

`init` walks you through installing an SSH key on the cluster (you type your
DTU password into OpenSSH's own prompt — this tool never sees it). After that,
the VPN is no longer needed for cluster work.

## Daily use

```
dtu-hpc up                 # start the control plane, authenticate once
dtu-hpc status             # check it
dtu-hpc audit              # see everything your agents did
dtu-hpc down               # revoke all agent access
```

Then talk to your agent:

> "Run the GRPO config on 2 A100s with a 4-hour budget, and chase it — if it
> OOMs, halve the batch size and resubmit."

### What this does NOT protect against

Being straight about the boundary, because the audience is students staking a
real account: **a batch job is arbitrary code running as you on a compute
node.** That is what a batch job is on every cluster. This tool bounds what
gets submitted — resources, destinations, rate — and records all of it. It does
not sandbox the code inside your job, and an agent that can submit jobs
effectively has your Unix account on the compute nodes. The audit log is a
plain file, so it is evidence against a *confused* agent, not a *hostile* one.
Run this with agents you'd trust with your cluster account, and read the log.

## Safety model in one paragraph

The daemon holds the only credential and the only connection. Agents reach it
through a token-gated loopback socket and can only invoke allowlisted
operations with regex-validated arguments. Writes are confined to declared
project roots, re-validated after the server resolves symlinks. Job scripts are
rendered *by the daemon* from a spec — agents never supply shell — and are
checked against ceilings held outside the project tree, so a profile an agent
can edit cannot raise them. Every operation, parameter, and denial is appended
to a local audit log. Killing one process revokes access.

## Status

Early but real: read, sync, submit, and chain paths all work and have been
exercised against DCC. Five independent code reviews (2026-07) found and fixed
a class of bugs where resource limits were enforced in the wrong process; see
DESIGN.md §2 for what changed and why.

Known gaps, honestly: there is no durable run ledger yet, so a run's story lives
across a manifest, the scheduler, and logs rather than one place. Human approval
above thresholds is currently asserted by the agent rather than confirmed
through a trusted channel. Artifact fetch (pulling results back) isn't built.
There are no tests yet.

Issues and PRs welcome — especially from other DTU students. If you run a
cluster and want a backend for it, `lsf.py` is the template to copy.

## For DCC / HPC support staff

This tool submits standard `bsub` jobs from a standard SSH session and runs
nothing on your infrastructure. It exists to make agent-assisted usage *more*
conservative than manual usage: validated scripts, capped resources, no
login-node compute, full local audit. We'd welcome your review of the
operation allowlist — open an issue or reach out.

## Troubleshooting

Login failing? Read [docs/AUTH.md](docs/AUTH.md) first. The short version:
outside the DTU VPN the cluster requires an SSH key **and** your password, so a
missing key looks exactly like a wrong password. Turn on the VPN, start the
daemon, press Enter at the passphrase prompt, type your DTU password.