vram-mcp

Give your AI agents a shared view of GPU memory. Through MCP, they can inspect what's using VRAM, make room for Ollama models, and see which sessions are already relying on them.

Each client runs its own server; sessions under the same OS user share claims, reservations, pending-operation records, and an audit log. Capacity and process readings apply to one configured GPU. Claimed or recently busy models are protected from eviction by default.

Get started

Use Python 3.10+, a running Ollama instance, and an NVIDIA GPU with drivers for memory readings. Windows and Linux are tested. Without nvidia-smi or NVML, model operations and claims still work, but some readings are unavailable.

Run vram-mcp on the machine hosting Ollama and the GPU. Changing OLLAMA_BASE_URL redirects model requests; GPU and process inspection always remain local. On a multi-GPU host, set VRAM_MCP_GPU_INDEX in every client's server environment to select the same device; the default is GPU index 0.

Connect your client

With uv and Git installed, choose your client below. uvx fetches vram-mcp from GitHub into an isolated environment; no source checkout is needed.

Claude Code:

claude mcp add vram --scope user -- uvx --from git+https://github.com/sushiHex/vram-mcp vram-mcp

Codex:

codex mcp add vram -- uvx --from git+https://github.com/sushiHex/vram-mcp vram-mcp

Hermes — merge into ~/.hermes/config.yaml:

mcp_servers:
  vram:
    command: uvx
    args: ["--from", "git+https://github.com/sushiHex/vram-mcp", "vram-mcp"]

Merge this entry into your client's MCP configuration:

{
  "mcpServers": {
    "vram": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/sushiHex/vram-mcp", "vram-mcp"]
    }
  }
}

If the client cannot find uvx, use its absolute executable path. In JSON, Windows paths need escaped backslashes, such as C:\\tools\\uvx.exe.

Reconnect your client after registration. It launches the server over stdio; running vram-mcp by hand waits for MCP input. Prefer a local installation? See installing from source.

Make your first check

Ask your agent:

Check GPU memory with vram-mcp. Show what's loaded, who is using it, and how much room is left. Report any unavailable readings.

The agent should call vram_status() and list_claims(). Status includes a readable summary, the selected GPU, loaded models, available process details, and observations metadata. Check each observation's status; unavailable means the corresponding value is unknown, rather than empty or zero. list_claims() also shows capacity reserved for training or other non-Ollama work.

Related MCP server: Ollama MCP Server

Agent workflow

These examples are MCP tool calls made inside a connected client, not shell commands or a Python API. Use a descriptive session label for owner and by, such as codex:review, so another agent can identify your work.

1. Inspect before changing memory. Call vram_status() and list_claims(). Model names are canonicalized consistently: a bare final name such as llama3 becomes llama3:latest; comparison is case-insensitive, and Ollama's default registry/library prefix is removed.

2. Claim a model before relying on it. Replace the example model below with one already installed in Ollama. Continue only if the claim succeeds, and save its returned claim_id.

   claim(model="llama3:latest", owner="codex:review", purpose="Review local code")

3. Make room when needed. ensure_free(gb=8, by="codex:review") unloads unprotected models largest-first. Choose the target for your workload and check outcome, free_mb, declined, reserved_mb, and observations. It does not evict when selected-GPU capacity or Ollama residency is unknown. Reaching the target does not give you ownership of that space.

4. Load and check. Call warm(model="llama3:latest", keep_alive="10m", by="codex:review") when needed. Inspect outcome, reason, and observations. succeeded means residency was reconciled after the request; refused means no Ollama request was sent; failed means Ollama definitively rejected it; and unknown means the request may still be running and includes pending_until. Do not retry a same-model mutation until that pending window ends. Warm admission is also serialized across models sharing the selected GPU. A model that was already resident has zero incremental residency cost (reason="already_resident"). Loading successfully does not guarantee full GPU residency. Run inference through your usual Ollama client.

5. Renew and release. Claims expire after one hour by default. Call renew(claim_id="<returned claim_id>") before expiry for longer work and release(claim_id="<returned claim_id>") when finished, including if loading fails. Releasing a claim does not unload the model; claim expiry and Ollama's keep_alive are separate.

For training or diffusion, use reserve(gb=8, owner="codex:training", purpose="LoRA training") to declare capacity, then renew/release its claim_id in the same way. A reservation records intent; it does not allocate memory.

Validation rejects invalid requests: capacities, TTLs, and trend windows must be finite and positive; model, owner, purpose, and caller labels must be nonblank; warm accepts a positive Ollama duration or -1 for indefinite residency. Use unload() rather than a zero keep_alive.

Coordination is cooperative. Direct Ollama calls can bypass it. force=True on unload, ensure_free, or warm overrides claims, busy protection, or reservation admission; use it only after resolving competing work. It does not bypass an unreadable coordination ledger or another pending mutation. busy=null means unknown and does not block eviction on its own. See coordination details.

Understand the readings

Missing required telemetry produces pressure.state="unknown". Read pressure.coverage and top-level observations to see which evidence was available. Driver-spill detection is best effort and requires non-local memory that can be attributed to the selected GPU; adapter-aggregated Windows counters do not establish that. CPU offload is tracked separately. Disabling auditing stops history detection and trend sampling, but does not disable current GPU or process observations.

Use history() to investigate a model disappearing and trend(hours=1) to review memory changes. Trends are sampled when vram_status() reads the GPU; there is no background sampler. See diagnostics.

Tools

The client's MCP schema provides full arguments and defaults.

Guides

Development

git clone https://github.com/sushiHex/vram-mcp.git
cd vram-mcp
python -m venv .venv

Activate with source .venv/bin/activate on Linux or .\.venv\Scripts\Activate.ps1 in PowerShell, then run:

python -m pip install -e ".[dev]"
python -m pytest -q

Tests use mocked GPU, process, and HTTP calls; no GPU or Ollama daemon is needed. CI covers Python 3.10–3.14 on Windows and Linux. Current model management targets Ollama and NVIDIA; AMD/Intel telemetry and vLLM/llama.cpp management are future work.

License

MIT — see LICENSE. Companion project: hardline-mcp for agent messaging.