vram-mcp
Use vram-mcp to inspect and coordinate shared GPU VRAM for Ollama through MCP.
Inspect the selected GPU, resident Ollama models, pressure, process details, claims, and observations with
vram_statusandlist_loaded.Free memory by unloading unprotected models to reach a target with
ensure_free, or evict a named model withunload(respecting claims/busy signals unless forced).Load or refresh Ollama models with keep-alive and admission checks via
warm, including reservation-aware and pending-operation outcomes.Coordinate cooperative use by claiming models or reserving GB capacity, then renewing, releasing, or listing claims/reservations.
Review audit history, free-VRAM trends, and configuration suggestions with
history,trend, andadvise.Note that coordination is cooperative: direct Ollama calls can bypass it, and
forcecannot override unreadable ledgers or pending mutations.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@vram-mcp Free up 2GB of VRAM"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
vram-mcp
Give your AI agents a shared view of GPU memory. Through MCP, they can inspect what's using VRAM, make room for Ollama models, and see which sessions are already relying on them.
Each client session runs its own server, of about 100 MB. Sessions under the
same OS user share claims, reservations, pending-operation records, and an
audit log, kept in ~/.cache/vram-mcp by default. Capacity and process
readings apply to one configured GPU. Claimed or recently busy models are
protected from eviction by default.
On Windows, vram-mcp also measures whether the driver is paging VRAM, so
pressure tells a GPU that is merely full from one that is thrashing. It also
names model runtimes on the GPU that the configured Ollama does not serve, such
as a separate llama-server.
Get started
Use Python 3.10+, a running Ollama instance, and an NVIDIA GPU with
drivers for memory readings. Windows and Linux are tested. Without
nvidia-smi or NVML, claims and unloads still work, but some readings are
unavailable, and warm needs force=True because no fit can be verified.
Run vram-mcp on the machine hosting Ollama and the GPU. Changing
OLLAMA_BASE_URL redirects model requests; GPU and process inspection always
remain local. On a multi-GPU host, set VRAM_MCP_GPU_INDEX in every client's
server environment to select the same device; the default is GPU index 0.
Connect your client
With uv and Git
installed, choose your client below. uvx fetches vram-mcp from GitHub into an
isolated environment; no source checkout is needed.
These commands install 0.3.1, the latest release. This page describes
main, which is ahead of it: paging detection on Windows, warm with
num_ctx, strict_fit and partial-fit reporting, unmonitored-runtime
reporting, and the Kokoro worker are unreleased. To run main, drop @v0.3.1
from the Git URL.
claude mcp add vram --scope user -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcpcodex mcp add vram -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcpHermes — merge into ~/.hermes/config.yaml:
mcp_servers:
vram:
command: uvx
args: ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]Merge this entry into your client's MCP configuration:
{
"mcpServers": {
"vram": {
"command": "uvx",
"args": ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]
}
}
}If the client cannot find uvx, use its absolute executable path. In JSON,
Windows paths need escaped backslashes, such as C:\\tools\\uvx.exe.
On Windows, the MCP SDK needs pywin32, which has no free-threaded build. If
uv picks a free-threaded Python (3.14t) and the install fails, add
--python 3.13 right after uvx.
Some clients start servers with a minimal environment that lacks
ProgramFiles. Under it, nvidia-smi fails with "Failed to initialize NVML:
Unknown Error", and every GPU reading is unavailable. Add ProgramFiles (for
example C:\Program Files) to the server's environment in the client
configuration. #52 makes
vram-mcp supply it.
Reconnect your client after registration. It launches the server over stdio;
running vram-mcp by hand waits for MCP input. Prefer a local installation?
See installing from source.
Upgrading from 0.2? Stop every client using vram-mcp first; old and new servers must never share one ledger. See the 0.3.0 notes and the 0.3.1 fix.
Make your first check
Ask your agent:
Check GPU memory with vram-mcp. Show what's loaded, who is using it, and how much room is left. Report any unavailable readings.
The agent should call vram_status() and list_claims(). Status includes a
readable summary, the selected GPU, loaded models, available process details
(command lines cut to 160 characters unless cmdline_chars says otherwise; see
configuration), and observations metadata. Check each observation's status; unavailable
means the corresponding value is unknown, rather than empty or zero.
list_claims() also shows capacity reserved for training or other non-Ollama
work.
Related MCP server: mcp-gpu-server
Agent workflow
These examples are MCP tool calls made inside a connected client, not shell
commands or a Python API. Use a descriptive session label for owner and by,
such as codex:review, so another agent can identify your work.
Inspect before changing memory. Call
vram_status()andlist_claims(). Model names are canonicalized consistently: a bare final name such asllama3becomesllama3:latest; comparison is case-insensitive, and Ollama's default registry/library prefix is removed.Claim a model before relying on it. Replace the example model below with one already installed in Ollama. Continue only if the claim succeeds, and save its returned
claim_id.claim(model="llama3:latest", owner="codex:review", purpose="Review local code")Make room when needed.
ensure_free(gb=8, by="codex:review")unloads unprotected models largest-first. Choose the target for your workload and checkoutcome,free_mb,declined,reserved_mb, andobservations. It does not evict when selected-GPU capacity or Ollama residency is unknown. Reaching the target does not give you ownership of that space.Load and check. Call
warm(model="llama3:latest", keep_alive="10m", by="codex:review")when needed. Inspectoutcome,reason, andobservations.succeededmeans residency was reconciled after the request;refusedmeans no Ollama request was sent;failedmeans Ollama definitively rejected it; andunknownmeans the request may still be running and includespending_until. Do not retry a same-model mutation until that pending window ends. Unknown free VRAM refuses a new load (reason="free_unknown") rather than guessing. Passnum_ctxto request a context length; admission then counts an estimated KV cache (kv_cache_mb), timesnum_parallelif the server runs parallel slots. Passstrict_fit=True(withnum_ctx) to also refuse a load that would not fit in free VRAM minus a 1 GiB margin, even when nothing is reserved;forcecannot override it. A load slower thantimeout_seconds(defaultVRAM_MCP_LOAD_TIMEOUT, 110 s) ends asunknownwithreason="load_timeout". Warm admission is also serialized across models sharing the selected GPU. A model that was already resident has zero incremental residency cost (reason="already_resident"). Loading successfully does not guarantee full GPU residency: when Ollama places part of the model outside VRAM, the result readspartial_fit: truewithoffloaded_mb, and the outcome stayssucceeded.partial_fitisnullwhen residency couldn't be read. vram-mcp reports a spill but never unloads or retries on its account. Every result also carrieselapsed_sand, when the request failed, theexceptionclass. The audit event keeps these, along with admission's inputs and the names of any unavailable sources. Run inference through your usual Ollama client.Renew and release. Claims expire after one hour by default. Call
renew(claim_id="<returned claim_id>")before expiry for longer work andrelease(claim_id="<returned claim_id>")when finished, including if loading fails. Releasing a claim does not unload the model; claim expiry and Ollama'skeep_aliveare separate.
For training or diffusion, use reserve(gb=8, owner="codex:training", purpose="LoRA training") to declare capacity, then renew/release its claim_id
in the same way. A reservation records intent; it does not allocate memory.
Validation rejects invalid requests: capacities, TTLs, and trend windows
must be finite and positive; model, owner, purpose, and caller labels must be
nonblank; warm accepts a positive Ollama duration or -1 for indefinite
residency. Use unload() rather than a zero keep_alive.
Coordination is cooperative. Direct Ollama calls can bypass it. force=True
on unload, ensure_free, or warm overrides claims, busy protection, or
reservation admission; use it only after resolving competing work. It does not
bypass an unreadable coordination ledger or another pending mutation.
busy=null means unknown and does not block eviction on its own. See
coordination details.
Understand the readings
| Meaning |
| Nothing is wrong, and every input that claim rests on was actually read. Missing evidence yields |
| Less than 1 GiB is free on the selected GPU. |
| Ollama placed part of a model on the CPU; expect slower inference. |
| The driver is paging VRAM: in at least 5 of the last 10 seconds, WDDM's memory manager kept the GPU busy. |
Missing required telemetry produces pressure.state="unknown". Read
pressure.coverage and top-level observations to see which evidence was
available. Paging detection is best effort. It needs a paging reading for the
selected GPU, which vram-mcp takes on Windows from a sampler that starts with
the server. On Linux no such reading exists, so pressure reads unknown
where it would read ok. The detection is a measured correlation, not a WDDM
contract; see the acceptance notes.
Non-local memory is reported but does not decide the verdict. CPU
offload is tracked separately.
vram_status lists, under unmonitored_runtimes, model runners (llama-server,
ollama_llama_server) on the selected GPU that no model in the configured
Ollama accounts for, and its summary names them. Their models are not in
loaded, and vram-mcp can't unload them. Check list_claims() before acting
on one: another session may own it. Disabling auditing
stops history detection and trend sampling, but does not disable current GPU or
process observations.
Use history() to investigate a model disappearing and trend(hours=1) to
review memory changes. Trends are sampled when vram_status() reads the GPU,
not in the background. The Windows paging sampler is separate: it only informs
pressure. See diagnostics.
Tools
The client's MCP schema provides full arguments and defaults.
Tool | Purpose |
| Inspect GPU memory, models, claims, activity, and pressure. |
| List resident models and their claim/busy details. |
| Try to reach a free-memory target by unloading unprotected models. |
| Evict a named model, respecting claims and recent activity. |
| Load a model for a chosen |
| Declare model use or capacity needed for other GPU work. |
| Extend or end your claim or reservation. |
| See active model claims and capacity reservations. |
| Review recorded actions, observed changes, and memory samples. |
| Get configuration suggestions; no settings are changed. |
| Optional Windows-only launch and settlement of the operator-configured reviewed Kokoro worker, with bounded exit diagnostics; launch acceptance is not GPU readiness. |
Guides
Guide | Read it for |
Source installation, environment variables, and connection troubleshooting. | |
Claims, reservations, protection limits, pressure, and audit history. | |
The opt-in run against a real GPU and Ollama, and the environments it has validated. | |
Optional private profile, supported worker pin, admission, process ownership and separate deployment/qualification limits. | |
What changed in each version, and how to upgrade. | |
How a GitHub Release builds, attests and publishes the distributions. |
Development
git clone https://github.com/sushiHex/vram-mcp.git
cd vram-mcp
python -m venv .venvActivate with source .venv/bin/activate on Linux or
.\.venv\Scripts\Activate.ps1 in PowerShell, then run:
python -m pip install -e ".[dev]"
python -m pytest -qTests use mocked GPU, process, and HTTP calls; no GPU or Ollama daemon is needed. Validation on real hardware is a separate, opt-in run; see hardware acceptance. CI covers Python 3.10–3.14 on Windows and Linux. Current model management targets Ollama and NVIDIA. Runners outside Ollama are reported, not managed; AMD/Intel telemetry and vLLM/llama.cpp management are future work. Contributors, human or agent, should read AGENTS.md for layout, style and testing conventions.
License
MIT — see LICENSE. Companion project: hardline-mcp for agent messaging.
Available Tools
13 toolsadviseB
Suggest env/config changes to keep VRAM healthy (heuristics).
Inputs: no named parameters.
claimA
Declare that you're using model for purpose.
Lets other sessions see who's using a model and why before deciding to
evict it. Renew before ttl_seconds elapses if still in use — an
un-renewed claim simply expires, so a crashed session never leaves a
permanently-stuck claim.
Inputs: model (required), owner (required), purpose (required), ttl_seconds. Returns: ok, model, detail, reason, +4 more
ensure_freeA
Free VRAM until at least gb GB is available. Skips claimed/busy models
unless force=True. by records the requester in the audit log.
Also reports reserved_mb — how much of the resulting free VRAM other
sessions have reserved for non-Ollama work (None if unreadable).
Inputs: gb (required), by, force. Returns: ok, model, detail, reason, +12 more
historyA
The VRAM audit trail, newest first: who ran unload/ensure_free/warm, and
which models/processes appeared or disappeared (with a best-effort cause).
Filter by model, type (action|disappeared|appeared), limit, or an
ISO since floor. Answers 'what happened to model X?'.
Inputs: type, limit, model, since.
list_claimsB
See who's claiming what right now (all models, or one).
Inputs: model. Returns: ok, claims, outcome, summary, +1 more
list_loadedA
List the models currently resident in VRAM, with claim/busy detail.
Inputs: no named parameters.
releaseB
Release a claim early, before its TTL would expire.
Inputs: claim_id (required). Returns: ok, model, detail, reason, +3 more
renewB
Extend an existing claim's expiry before it lapses.
Inputs: claim_id (required), ttl_seconds. Returns: ok, model, detail, reason, +4 more
reserveA
Reserve gb GB of VRAM — a claim on capacity, not on a named model.
Use this for non-Ollama GPU work (a training run, a diffusion job) so other
sessions can see the VRAM is spoken for. pid is advisory. Reservations
expire by TTL like claims, so a crashed session never leaves one stuck.
COOPERATIVE: this gates vram-mcp's own warm(), but vram-mcp cannot
intercept an Ollama auto-load triggered by a direct /api/generate call
from another process.
Inputs: gb (required), owner (required), purpose (required), pid, +1 more. Returns: gb, ok, model, detail, +5 more
trendA
Free-VRAM trend over the last hours, from the sampled audit log.
Answers "was this a gradual erosion or a sudden spike?" — the question a
point-in-time vram_status() cannot. Returns direction, min/max/latest
free MB (latest is null when the newest sample carries no reading),
how many samples showed driver spill, and the raw samples.
samples holds at most the 200 most recent rows so a long window can't
flood the caller's context; samples_truncated says whether older rows
were dropped. Every summary figure is computed over the FULL window either
way.
Inputs: hours.
unloadA
Evict a single model from VRAM now (Ollama keep_alive=0).
Refuses by default if model has an active claim or a best-effort busy
signal — pass force=True to override (busy is windowed and can lag a few
seconds past a generation). by records who requested the eviction in the
audit log (see history).
Inputs: model (required), by, force. Returns: ok, busy, model, scope, +23 more
vram_statusA
Report the selected GPU and server-wide Ollama model residency.
Includes device identity, source health/timestamps/coverage, claims, recent
GPU activity, and pressure (ok|tight|degraded|thrashing|unknown). Unavailable
residency is null; unavailable telemetry never establishes an empty or
healthy GPU. thrashing means the driver was measured paging VRAM.
Each other_processes command line is cut to its first cmdline_chars
characters (default 160), ending in "…" when cut. 0 omits the field;
null returns command lines whole.
Inputs: cmdline_chars.
warmA
Load a model or refresh its keep-alive (e.g. "5m"; "-1" pins indefinitely).
An already resident model requires zero additional capacity. New loads account for reservations using an approximate model size; size_verified describes that estimate, while outcome describes verified residency. Returns succeeded|refused|failed|unknown. On unknown, inspect residency and pending_until before retrying. force=True bypasses admission checks but cannot override a pending operation. Zero durations must use unload().
Inputs: model (required), by, force, keep_alive. Returns: ok, busy, model, scope, +23 more
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 13 tools
Most tools have clearly distinct purposes, but there is real overlap between list_loaded and vram_status (both report residency) and between history and trend (both surface historical data). The descriptions do enough to separate them, so an agent can mostly disambiguate, but these pairs require reading carefully.
There is a discernible pattern — verb_noun for actions (list_loaded, ensure_free, list_claims) and bare verbs for mutations (unload, warm, renew, release) — but query tools use bare nouns (history, trend, advise, claim, reserve), mixing conventions. It is readable but not a single predictable scheme.
13 tools is well-scoped for VRAM lifecycle management: monitoring, eviction/loading, and a cooperative claim/reservation system. Each tool earns its place without redundancy or padding.
The claim lifecycle is complete (claim/renew/release/list_claims) and monitoring covers status, history, trend, and advice. Minor gap: reservations (reserve) lack an explicit release/list counterpart, so non-Ollama capacity reservations may be harder to inspect or cancel.
Maintenance
Related MCP Connectors
Rent cloud GPUs from AI agents: live prices, VRAM estimates, launch, logs and stop.
Can this LLM run on my GPU? VRAM, speed ceiling and what fits instead, for any model and GPU.
On-demand GPU nodes for agents: create nodes, run commands, and submit jobs, billed by the minute.
Agent GPU compute scheduling: compare billing, approve budgets, run tasks and persist results.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables complete local Ollama management including listing models, chatting with local LLMs, starting/stopping the server, and getting intelligent model recommendations for specific tasks through natural language commands.94MIT
- AlicenseAqualityBmaintenanceExposes NVIDIA GPU metrics (info, utilization, VRAM, temperature) via MCP tools for real-time querying from AI assistants.5MIT
- AlicenseAqualityCmaintenanceEnables AI agents to search Vast.ai GPU marketplace, deploy compute-heavy video models or ComfyUI, and guarantee instances are torn down after the job finishes.171MIT
- AlicenseAqualityCmaintenanceEnables AI agents to search, compare, and right-size Hugging Face models, including estimating whether a model fits in available GPU VRAM before downloading it.535 PyPIMIT