Skip to main content
Glama

vram-mcp

Give your AI agents a shared view of GPU memory. Through MCP, they can inspect what's using VRAM, make room for Ollama models, and see which sessions are already relying on them.

Each client session runs its own server, of about 100 MB. Sessions under the same OS user share claims, reservations, pending-operation records, and an audit log, kept in ~/.cache/vram-mcp by default. Capacity and process readings apply to one configured GPU. Claimed or recently busy models are protected from eviction by default.

On Windows, vram-mcp also measures whether the driver is paging VRAM, so pressure tells a GPU that is merely full from one that is thrashing. It also names model runtimes on the GPU that the configured Ollama does not serve, such as a separate llama-server.

Get started

Use Python 3.10+, a running Ollama instance, and an NVIDIA GPU with drivers for memory readings. Windows and Linux are tested. Without nvidia-smi or NVML, claims and unloads still work, but some readings are unavailable, and warm needs force=True because no fit can be verified.

Run vram-mcp on the machine hosting Ollama and the GPU. Changing OLLAMA_BASE_URL redirects model requests; GPU and process inspection always remain local. On a multi-GPU host, set VRAM_MCP_GPU_INDEX in every client's server environment to select the same device; the default is GPU index 0.

Connect your client

With uv and Git installed, choose your client below. uvx fetches vram-mcp from GitHub into an isolated environment; no source checkout is needed.

These commands install 0.3.1, the latest release. This page describes main, which is ahead of it: paging detection on Windows, warm with num_ctx, strict_fit and partial-fit reporting, unmonitored-runtime reporting, and the Kokoro worker are unreleased. To run main, drop @v0.3.1 from the Git URL.

Claude Code:

claude mcp add vram --scope user -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcp

Codex:

codex mcp add vram -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcp

Hermes — merge into ~/.hermes/config.yaml:

mcp_servers:
  vram:
    command: uvx
    args: ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]

Merge this entry into your client's MCP configuration:

{
  "mcpServers": {
    "vram": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]
    }
  }
}

If the client cannot find uvx, use its absolute executable path. In JSON, Windows paths need escaped backslashes, such as C:\\tools\\uvx.exe.

On Windows, the MCP SDK needs pywin32, which has no free-threaded build. If uv picks a free-threaded Python (3.14t) and the install fails, add --python 3.13 right after uvx.

Some clients start servers with a minimal environment that lacks ProgramFiles. Under it, nvidia-smi fails with "Failed to initialize NVML: Unknown Error", and every GPU reading is unavailable. Add ProgramFiles (for example C:\Program Files) to the server's environment in the client configuration. #52 makes vram-mcp supply it.

Reconnect your client after registration. It launches the server over stdio; running vram-mcp by hand waits for MCP input. Prefer a local installation? See installing from source.

Upgrading from 0.2? Stop every client using vram-mcp first; old and new servers must never share one ledger. See the 0.3.0 notes and the 0.3.1 fix.

Make your first check

Ask your agent:

Check GPU memory with vram-mcp. Show what's loaded, who is using it, and how much room is left. Report any unavailable readings.

The agent should call vram_status() and list_claims(). Status includes a readable summary, the selected GPU, loaded models, available process details (command lines cut to 160 characters unless cmdline_chars says otherwise; see configuration), and observations metadata. Check each observation's status; unavailable means the corresponding value is unknown, rather than empty or zero. list_claims() also shows capacity reserved for training or other non-Ollama work.

Related MCP server: mcp-gpu-server

Agent workflow

These examples are MCP tool calls made inside a connected client, not shell commands or a Python API. Use a descriptive session label for owner and by, such as codex:review, so another agent can identify your work.

  1. Inspect before changing memory. Call vram_status() and list_claims(). Model names are canonicalized consistently: a bare final name such as llama3 becomes llama3:latest; comparison is case-insensitive, and Ollama's default registry/library prefix is removed.

  2. Claim a model before relying on it. Replace the example model below with one already installed in Ollama. Continue only if the claim succeeds, and save its returned claim_id.

    claim(model="llama3:latest", owner="codex:review", purpose="Review local code")
  3. Make room when needed. ensure_free(gb=8, by="codex:review") unloads unprotected models largest-first. Choose the target for your workload and check outcome, free_mb, declined, reserved_mb, and observations. It does not evict when selected-GPU capacity or Ollama residency is unknown. Reaching the target does not give you ownership of that space.

  4. Load and check. Call warm(model="llama3:latest", keep_alive="10m", by="codex:review") when needed. Inspect outcome, reason, and observations. succeeded means residency was reconciled after the request; refused means no Ollama request was sent; failed means Ollama definitively rejected it; and unknown means the request may still be running and includes pending_until. Do not retry a same-model mutation until that pending window ends. Unknown free VRAM refuses a new load (reason="free_unknown") rather than guessing. Pass num_ctx to request a context length; admission then counts an estimated KV cache (kv_cache_mb), times num_parallel if the server runs parallel slots. Pass strict_fit=True (with num_ctx) to also refuse a load that would not fit in free VRAM minus a 1 GiB margin, even when nothing is reserved; force cannot override it. A load slower than timeout_seconds (default VRAM_MCP_LOAD_TIMEOUT, 110 s) ends as unknown with reason="load_timeout". Warm admission is also serialized across models sharing the selected GPU. A model that was already resident has zero incremental residency cost (reason="already_resident"). Loading successfully does not guarantee full GPU residency: when Ollama places part of the model outside VRAM, the result reads partial_fit: true with offloaded_mb, and the outcome stays succeeded. partial_fit is null when residency couldn't be read. vram-mcp reports a spill but never unloads or retries on its account. Every result also carries elapsed_s and, when the request failed, the exception class. The audit event keeps these, along with admission's inputs and the names of any unavailable sources. Run inference through your usual Ollama client.

  5. Renew and release. Claims expire after one hour by default. Call renew(claim_id="<returned claim_id>") before expiry for longer work and release(claim_id="<returned claim_id>") when finished, including if loading fails. Releasing a claim does not unload the model; claim expiry and Ollama's keep_alive are separate.

For training or diffusion, use reserve(gb=8, owner="codex:training", purpose="LoRA training") to declare capacity, then renew/release its claim_id in the same way. A reservation records intent; it does not allocate memory.

Validation rejects invalid requests: capacities, TTLs, and trend windows must be finite and positive; model, owner, purpose, and caller labels must be nonblank; warm accepts a positive Ollama duration or -1 for indefinite residency. Use unload() rather than a zero keep_alive.

Coordination is cooperative. Direct Ollama calls can bypass it. force=True on unload, ensure_free, or warm overrides claims, busy protection, or reservation admission; use it only after resolving competing work. It does not bypass an unreadable coordination ledger or another pending mutation. busy=null means unknown and does not block eviction on its own. See coordination details.

Understand the readings

pressure.state

Meaning

ok

Nothing is wrong, and every input that claim rests on was actually read. Missing evidence yields unknown, never ok.

tight

Less than 1 GiB is free on the selected GPU.

degraded

Ollama placed part of a model on the CPU; expect slower inference.

thrashing

The driver is paging VRAM: in at least 5 of the last 10 seconds, WDDM's memory manager kept the GPU busy.

Missing required telemetry produces pressure.state="unknown". Read pressure.coverage and top-level observations to see which evidence was available. Paging detection is best effort. It needs a paging reading for the selected GPU, which vram-mcp takes on Windows from a sampler that starts with the server. On Linux no such reading exists, so pressure reads unknown where it would read ok. The detection is a measured correlation, not a WDDM contract; see the acceptance notes. Non-local memory is reported but does not decide the verdict. CPU offload is tracked separately.

vram_status lists, under unmonitored_runtimes, model runners (llama-server, ollama_llama_server) on the selected GPU that no model in the configured Ollama accounts for, and its summary names them. Their models are not in loaded, and vram-mcp can't unload them. Check list_claims() before acting on one: another session may own it. Disabling auditing stops history detection and trend sampling, but does not disable current GPU or process observations.

Use history() to investigate a model disappearing and trend(hours=1) to review memory changes. Trends are sampled when vram_status() reads the GPU, not in the background. The Windows paging sampler is separate: it only informs pressure. See diagnostics.

Tools

The client's MCP schema provides full arguments and defaults.

Tool

Purpose

vram_status

Inspect GPU memory, models, claims, activity, and pressure.

list_loaded

List resident models and their claim/busy details.

ensure_free

Try to reach a free-memory target by unloading unprotected models.

unload

Evict a named model, respecting claims and recent activity.

warm

Load a model for a chosen keep_alive, checking reservations.

claim, reserve

Declare model use or capacity needed for other GPU work.

renew, release

Extend or end your claim or reservation.

list_claims

See active model claims and capacity reservations.

history, trend

Review recorded actions, observed changes, and memory samples.

advise

Get configuration suggestions; no settings are changed.

load_kokoro, kokoro_status, stop_kokoro

Optional Windows-only launch and settlement of the operator-configured reviewed Kokoro worker, with bounded exit diagnostics; launch acceptance is not GPU readiness.

Guides

Guide

Read it for

Configuration

Source installation, environment variables, and connection troubleshooting.

Coordination and diagnostics

Claims, reservations, protection limits, pressure, and audit history.

Hardware acceptance

The opt-in run against a real GPU and Ollama, and the environments it has validated.

Owned Kokoro worker

Optional private profile, supported worker pin, admission, process ownership and separate deployment/qualification limits.

Release notes

What changed in each version, and how to upgrade.

Releasing

How a GitHub Release builds, attests and publishes the distributions.

Development

git clone https://github.com/sushiHex/vram-mcp.git
cd vram-mcp
python -m venv .venv

Activate with source .venv/bin/activate on Linux or .\.venv\Scripts\Activate.ps1 in PowerShell, then run:

python -m pip install -e ".[dev]"
python -m pytest -q

Tests use mocked GPU, process, and HTTP calls; no GPU or Ollama daemon is needed. Validation on real hardware is a separate, opt-in run; see hardware acceptance. CI covers Python 3.10–3.14 on Windows and Linux. Current model management targets Ollama and NVIDIA. Runners outside Ollama are reported, not managed; AMD/Intel telemetry and vLLM/llama.cpp management are future work. Contributors, human or agent, should read AGENTS.md for layout, style and testing conventions.

License

MIT — see LICENSE. Companion project: hardline-mcp for agent messaging.

Available Tools

13 tools
adviseB

Suggest env/config changes to keep VRAM healthy (heuristics).

Inputs: no named parameters.

claimA

Declare that you're using model for purpose.

Lets other sessions see who's using a model and why before deciding to evict it. Renew before ttl_seconds elapses if still in use — an un-renewed claim simply expires, so a crashed session never leaves a permanently-stuck claim.

Inputs: model (required), owner (required), purpose (required), ttl_seconds. Returns: ok, model, detail, reason, +4 more

ensure_freeA

Free VRAM until at least gb GB is available. Skips claimed/busy models unless force=True. by records the requester in the audit log.

Also reports reserved_mb — how much of the resulting free VRAM other sessions have reserved for non-Ollama work (None if unreadable).

Inputs: gb (required), by, force. Returns: ok, model, detail, reason, +12 more

historyA

The VRAM audit trail, newest first: who ran unload/ensure_free/warm, and which models/processes appeared or disappeared (with a best-effort cause). Filter by model, type (action|disappeared|appeared), limit, or an ISO since floor. Answers 'what happened to model X?'.

Inputs: type, limit, model, since.

list_claimsB

See who's claiming what right now (all models, or one).

Inputs: model. Returns: ok, claims, outcome, summary, +1 more

list_loadedA

List the models currently resident in VRAM, with claim/busy detail.

Inputs: no named parameters.

releaseB

Release a claim early, before its TTL would expire.

Inputs: claim_id (required). Returns: ok, model, detail, reason, +3 more

renewB

Extend an existing claim's expiry before it lapses.

Inputs: claim_id (required), ttl_seconds. Returns: ok, model, detail, reason, +4 more

reserveA

Reserve gb GB of VRAM — a claim on capacity, not on a named model.

Use this for non-Ollama GPU work (a training run, a diffusion job) so other sessions can see the VRAM is spoken for. pid is advisory. Reservations expire by TTL like claims, so a crashed session never leaves one stuck.

COOPERATIVE: this gates vram-mcp's own warm(), but vram-mcp cannot intercept an Ollama auto-load triggered by a direct /api/generate call from another process.

Inputs: gb (required), owner (required), purpose (required), pid, +1 more. Returns: gb, ok, model, detail, +5 more

trendA

Free-VRAM trend over the last hours, from the sampled audit log.

Answers "was this a gradual erosion or a sudden spike?" — the question a point-in-time vram_status() cannot. Returns direction, min/max/latest free MB (latest is null when the newest sample carries no reading), how many samples showed driver spill, and the raw samples.

samples holds at most the 200 most recent rows so a long window can't flood the caller's context; samples_truncated says whether older rows were dropped. Every summary figure is computed over the FULL window either way.

Inputs: hours.

unloadA

Evict a single model from VRAM now (Ollama keep_alive=0).

Refuses by default if model has an active claim or a best-effort busy signal — pass force=True to override (busy is windowed and can lag a few seconds past a generation). by records who requested the eviction in the audit log (see history).

Inputs: model (required), by, force. Returns: ok, busy, model, scope, +23 more

vram_statusA

Report the selected GPU and server-wide Ollama model residency.

Includes device identity, source health/timestamps/coverage, claims, recent GPU activity, and pressure (ok|tight|degraded|thrashing|unknown). Unavailable residency is null; unavailable telemetry never establishes an empty or healthy GPU. thrashing means the driver was measured paging VRAM.

Each other_processes command line is cut to its first cmdline_chars characters (default 160), ending in "…" when cut. 0 omits the field; null returns command lines whole.

Inputs: cmdline_chars.

warmA

Load a model or refresh its keep-alive (e.g. "5m"; "-1" pins indefinitely).

An already resident model requires zero additional capacity. New loads account for reservations using an approximate model size; size_verified describes that estimate, while outcome describes verified residency. Returns succeeded|refused|failed|unknown. On unknown, inspect residency and pending_until before retrying. force=True bypasses admission checks but cannot override a pending operation. Zero durations must use unload().

Inputs: model (required), by, force, keep_alive. Returns: ok, busy, model, scope, +23 more

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 13 tools

    Disambiguation4/5

    Most tools have clearly distinct purposes, but there is real overlap between list_loaded and vram_status (both report residency) and between history and trend (both surface historical data). The descriptions do enough to separate them, so an agent can mostly disambiguate, but these pairs require reading carefully.

    Naming Consistency3/5

    There is a discernible pattern — verb_noun for actions (list_loaded, ensure_free, list_claims) and bare verbs for mutations (unload, warm, renew, release) — but query tools use bare nouns (history, trend, advise, claim, reserve), mixing conventions. It is readable but not a single predictable scheme.

    Tool Count5/5

    13 tools is well-scoped for VRAM lifecycle management: monitoring, eviction/loading, and a cooperative claim/reservation system. Each tool earns its place without redundancy or padding.

    Completeness4/5

    The claim lifecycle is complete (claim/renew/release/list_claims) and monitoring covers status, history, trend, and advice. Minor gap: reservations (reserve) lack an explicit release/list counterpart, so non-Ollama capacity reservations may be harder to inspect or cancel.

    Maintenance

    ActivityActive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers