vram-mcp

An MCP server that lets AI agents share one NVIDIA GPU safely. Built for the reality of several Claude Code / agent sessions juggling Ollama models on a single card: before loading the next model, an agent can see exactly what's holding VRAM, who's using it and why, whether it's computing right now — and free space without stepping on another session's in-flight work.

• Claims — sessions declare who is using a model and why, in a shared crash-safe ledger. TTL-based: a killed session never leaves a stuck claim.

• Real busy detection — windowed per-process GPU utilization via NVML, with zero changes to how anything calls Ollama.

• Protected eviction — unload/ensure_free refuse to evict a claimed or actively-computing model by default; force=True when you've decided.

• Full visibility — every VRAM-holding process on the GPU, not just Ollama models, plus CPU-offload detection (size_vram < size = spilled to RAM).

• Pressure detection — distinguishes driver-forced VRAM spill to system RAM (the severe slow-mode) from Ollama's deliberate CPU offload.

• Degrades gracefully — no nvidia-smi/NVML/wmic? Readings become unknown/null, never wrong; model list / unload / warm keep working.

• NVIDIA + Ollama for now (see Roadmap).

Tools

Related MCP server: Ollama MCP Server

Requirements

• Ollama running locally (or reachable via OLLAMA_BASE_URL).

• NVIDIA GPU + drivers for VRAM numbers. nvidia-smi is optional — without it, VRAM is reported as unknown and model operations still function.

• Python 3.10+.

Install

Run directly from GitHub with uv (no install needed; the package is not on PyPI):

uvx --from git+https://github.com/sushiHex/vram-mcp vram-mcp

Or install from source for development:

git clone https://github.com/sushiHex/vram-mcp
cd vram-mcp
pip install -e .

Run

vram-mcp

The server speaks MCP over stdio, so it is normally launched by an MCP client rather than by hand.

MCP client config

Add this to your MCP client's mcpServers config (e.g. Claude Code / Claude Desktop):

{
  "mcpServers": {
    "vram": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/sushiHex/vram-mcp", "vram-mcp"]
    }
  }
}

Or, with a source checkout installed via pip install -e ., point command directly at the installed vram-mcp script.

Configuration

• OLLAMA_BASE_URL — Ollama endpoint. Defaults to http://127.0.0.1:11434.

• VRAM_MCP_AUDIT — disappearance detection + the perf-counter-based process table (other_processes size/name/cmdline on Windows). Defaults on; set to "0" to disable and skip the ~1 s Windows perf-counter cost on every vram_status()/list_loaded() call. It turns off four things, not one: disappearance/appearance detection, the process table, driver-spill detection (pressure.unexplained_spill_mb/spilling/thrashing — only the perf-counter table reports non-local VRAM, so pressure can still say degraded/tight/ok but never thrashing), and trend sampling (no samples are recorded, so trend() reports nothing over that period). Action events (unload/ensure_free/warm) are recorded either way.

• VRAM_MCP_MEANINGFUL_MB — minimum dedicated VRAM (MB) for a non-Ollama process to be tracked by disappearance detection. Defaults to 512.

• VRAM_MCP_EVENT_CAP — maximum events retained in events.jsonl (oldest pruned first). Defaults to 5000.

• VRAM_MCP_SAMPLE_SECONDS — minimum seconds between free-VRAM trend samples (shared across sessions). Defaults to 60. The throttle matters: the event log is capped, so unthrottled samples would evict the action and disappearance events that carry the real diagnostic value.

• VRAM_MCP_SPILL_MB — the floor of unexplained non-local VRAM (MB) below which pressure will not report spilling/thrashing. Defaults to 256, beneath which non-local usage is ordinary desktop noise (compositor, browser) rather than a model paging to system RAM. Raise it if a background app keeps a steady spill you don't care about; lower it to catch a spill earlier.

  Clearing the floor is necessary but not sufficient: the spill must also exceed the free VRAM. The driver evicts only when it runs out of room, so non-local memory on a card with gigabytes free is routine allocation, not paging — and "free VRAM or reduce load" would be advice about VRAM you already have. Both gates must pass, so raising this value can only ever silence the alarm, never trigger one. When free VRAM is unreadable (no nvidia-smi) the card cannot be cleared of blame and the floor decides alone.

  "Unexplained" is load-bearing on Windows/WDDM: a llama.cpp runner's deliberately CPU-offloaded layers are reported as that process's Non Local Usage, so pressure attributes each process's non-local memory before judging it. A runner is explained up to its own offload (total_size_mb - size_vram_mb, reported as explained_offload_mb); anything beyond that, plus every other process's non-local memory, is genuine driver paging (unexplained_spill_mb). non_local_mb remains the raw total of the two. A 32B model deliberately part-offloaded on a 24 GB card therefore reads degraded, not thrashing — while that same runner being paged because another app ballooned still reads thrashing.

Multi-session coordination

Since every session runs its own vram-mcp process, coordination happens via:

• Claims — a shared, file-based ledger (~/.cache/vram-mcp/claims.json) recording who's using a model and why. Call claim() when you start relying on a model; renew() periodically if still in use. An un-renewed claim simply expires — no cleanup needed if your session ends unexpectedly.

• Busy detection — best-effort, via NVML's per-process GPU utilization (not point-in-time; reads a short recent window so brief gaps between tokens don't misread as idle). Requires no changes to how you call Ollama — it's entirely on vram-mcp's side.

• Protection — unload()/ensure_free() refuse to evict a model that's claimed OR busy, by default. Pass force=True when you've already decided it's worth it.

Requires the nvidia-ml-py dependency (installed automatically). Falls back gracefully — claims/busy report as empty/null — on non-NVIDIA GPUs or if NVML is unavailable.

Reservations

reserve(gb, owner, purpose) claims capacity rather than a named model, for the GPU work vram-mcp can't otherwise see — a training run, a diffusion job. Reservations share the claim ledger, so they inherit the same TTL and crash-safety semantics, and ensure_free() reports how much of the free VRAM they account for.

Reservations are cooperative, not enforced. They do two things: gate vram-mcp's own warm() (which refuses when a reservation leaves no headroom for the model, overridable with force=True) and tell other sessions the VRAM is spoken for. vram-mcp cannot intercept an Ollama auto-load triggered by a direct /api/generate call from another process — nothing outside vram-mcp is obliged to look at the ledger.

Audit trail

Every session shares one append-only, bounded log at ~/.cache/vram-mcp/events.jsonl (paired with a ~/.cache/vram-mcp/last_seen.json baseline). Three kinds of events land there:

• Actions — every unload/ensure_free/warm call, tagged with the by argument you passed (defaults to "unknown" if omitted), whether it succeeded/failed/was refused, and why.

• Disappearances/appearances — on every vram_status()/list_loaded() call, vram-mcp diffs the current set of "meaningful" VRAM holders (every Ollama model, plus any other process using at least VRAM_MCP_MEANINGFUL_MB of dedicated VRAM) against the previous snapshot and logs what changed, with a best-effort cause:

  • self_action — a recent unload/ensure_free action from this log explains the disappearance.

  • external — no matching vram-mcp action; likely Ollama idle-expiry, memory-pressure eviction, or an unload issued outside vram-mcp.

  • unattributed — a non-Ollama process disappeared; vram-mcp can't observe why a process exited.

• Samples — a free-VRAM datapoint recorded on the same status calls, at most once per VRAM_MCP_SAMPLE_SECONDS across all sessions. trend(hours) reduces them to a direction, so you can tell a gradual erosion from a sudden spike — the question a point-in-time vram_status() can't answer. Only calls that actually read the GPU are sampled (list_loaded() skips the nvidia-smi spawn, so it never records one), and trend() returns at most the 200 most recent raw samples — flagged with samples_truncated — while its min/max/latest/direction figures always cover the full window.

Call history(model=None, type=None, limit=50, since=None) to query it — e.g. "what happened to llama3?" or "did anything unexpectedly vanish in the last hour?". The whole audit path is best-effort: a failure to read or write the log never breaks a tool call. Disable detection (and its ~1 s Windows perf-counter cost) with VRAM_MCP_AUDIT=0; tune retention with VRAM_MCP_EVENT_CAP and the size threshold with VRAM_MCP_MEANINGFUL_MB. Action events (unload/ensure_free/warm) are always recorded regardless of VRAM_MCP_AUDIT. Everything passive is gated by it, though — setting it to 0 disables the disappearance/appearance detection, the perf-counter process table, driver-spill detection (pressure can no longer reach thrashing, since only that table reports non-local VRAM) and trend sampling (nothing is recorded, so trend() has nothing to report for that period).

other_processes (from vram_status()) now carries real size_mb/name/ cmdline for every VRAM-holding process, not just Ollama's — on Windows via a \GPU Process Memory(*)\Dedicated Usage perf-counter sample joined with Get-CimInstance Win32_Process (the same source Task Manager uses), since NVML alone reports null sizes under WDDM. That perf-counter sample costs roughly 1 second per vram_status()/list_loaded() call; set VRAM_MCP_AUDIT=0 to skip it (falls back to NVML's raw process list, sizes included where NVML can report them, names/cmdlines omitted).

Development

pip install -e .
python -m pytest -q

The logic modules (gpu.py, ollama.py, core.py, nvml.py, ollama_correlate.py, claims.py) are free of any mcp import and are fully unit-tested with mocks — no real GPU, Ollama daemon, or mcp package required to run the test suite.

Roadmap

• Other backends: AMD (ROCm/rocm-smi), Intel (xpu-smi).

• Other runtimes: vLLM, llama.cpp.

License

MIT © 2026 sushiHex