# vram-mcp

Give your AI agents a shared view of GPU memory. Through
[MCP](https://modelcontextprotocol.io), they can inspect what's using VRAM,
make room for [Ollama](https://ollama.com) models, and see which sessions are
already relying on them.

Each client runs its own server; sessions under the same OS user share claims,
reservations, pending-operation records, and an audit log. Capacity and process
readings apply to one configured GPU. Claimed or recently busy models are
protected from eviction by default.

## Get started

Use **Python 3.10+**, a running **Ollama** instance, and an **NVIDIA GPU with
drivers** for memory readings. Windows and Linux are tested. Without
`nvidia-smi` or NVML, model operations and claims still work, but some readings
are unavailable.

Run vram-mcp on the machine hosting Ollama and the GPU. Changing
`OLLAMA_BASE_URL` redirects model requests; GPU and process inspection always
remain local. On a multi-GPU host, set `VRAM_MCP_GPU_INDEX` in every client's
server environment to select the same device; the default is GPU index `0`.

### Connect your client

With [uv](https://docs.astral.sh/uv/getting-started/installation/) and Git
installed, choose your client below. `uvx` fetches vram-mcp from GitHub into an
isolated environment; no source checkout is needed.

**[Claude Code](https://code.claude.com/docs/en/mcp):**

```sh
claude mcp add vram --scope user -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcp
```

**[Codex](https://learn.chatgpt.com/docs/extend/mcp?surface=cli):**

```sh
codex mcp add vram -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcp
```

**[Hermes](https://hermes-agent.nousresearch.com/docs/user-guide/features/mcp)** — merge into `~/.hermes/config.yaml`:

```yaml
mcp_servers:
  vram:
    command: uvx
    args: ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]
```

<details>
<summary>Claude Desktop and other clients using mcpServers JSON</summary>

Merge this entry into your client's MCP configuration:

```json
{
  "mcpServers": {
    "vram": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]
    }
  }
}
```

If the client cannot find `uvx`, use its absolute executable path. In JSON,
Windows paths need escaped backslashes, such as `C:\\tools\\uvx.exe`.

</details>

On Windows, the MCP SDK needs `pywin32`, which has no free-threaded build. If
`uv` picks a free-threaded Python (`3.14t`) and the install fails, add
`--python 3.13` right after `uvx`.

Reconnect your client after registration. It launches the server over stdio;
running `vram-mcp` by hand waits for MCP input. Prefer a local installation?
See [installing from source](docs/configuration.md#installing-from-source).

**Upgrading from 0.2?** Stop every client using vram-mcp first; old and new
servers must never share one ledger. See the [0.3.0 notes](docs/releases/v0.3.0.md)
and the [0.3.1 fix](docs/releases/v0.3.1.md).

### Make your first check

Ask your agent:

> Check GPU memory with vram-mcp. Show what's loaded, who is using it, and how
> much room is left. Report any unavailable readings.

The agent should call `vram_status()` and `list_claims()`. Status includes a
readable `summary`, the selected GPU, loaded models, available process details
(command lines cut to 160 characters unless `cmdline_chars` says otherwise; see
[configuration](docs/configuration.md)), and `observations` metadata. Check each observation's `status`; `unavailable`
means the corresponding value is unknown, rather than empty or zero.
`list_claims()` also shows capacity reserved for training or other non-Ollama
work.

## Agent workflow

These examples are **MCP tool calls** made inside a connected client, not shell
commands or a Python API. Use a descriptive session label for `owner` and `by`,
such as `codex:review`, so another agent can identify your work.

1. **Inspect before changing memory.** Call `vram_status()` and `list_claims()`.
   Model names are canonicalized consistently: a bare final name such as
   `llama3` becomes `llama3:latest`; comparison is case-insensitive, and
   Ollama's default registry/library prefix is removed.
2. **Claim a model before relying on it.** Replace the example model below
   with one already installed in Ollama. Continue only if the claim succeeds,
   and save its returned `claim_id`.

   ```text
   claim(model="llama3:latest", owner="codex:review", purpose="Review local code")
   ```

3. **Make room when needed.** `ensure_free(gb=8, by="codex:review")` unloads
   unprotected models largest-first. Choose the target for your workload and
   check `outcome`, `free_mb`, `declined`, `reserved_mb`, and `observations`.
   It does not evict when selected-GPU capacity or Ollama residency is unknown.
   Reaching the target does not give you ownership of that space.
4. **Load and check.** Call
   `warm(model="llama3:latest", keep_alive="10m", by="codex:review")` when needed.
   Inspect `outcome`, `reason`, and `observations`. `succeeded` means residency
   was reconciled after the request; `refused` means no Ollama request was sent;
   `failed` means Ollama definitively rejected it; and `unknown` means the
   request may still be running and includes `pending_until`. Do not retry a
   same-model mutation until that pending window ends. Warm admission is also
   serialized across models sharing the selected GPU. A model that was already
   resident has zero incremental residency cost (`reason="already_resident"`).
   Loading successfully does not guarantee full GPU residency. Run inference
   through your usual Ollama client.
5. **Renew and release.** Claims expire after one hour by default. Call
   `renew(claim_id="<returned claim_id>")` before expiry for longer work and
   `release(claim_id="<returned claim_id>")` when finished, including if loading
   fails. Releasing a claim does not unload the model; claim expiry and Ollama's
   `keep_alive` are separate.

For training or diffusion, use `reserve(gb=8, owner="codex:training",
purpose="LoRA training")` to declare capacity, then renew/release its `claim_id`
in the same way. A reservation records intent; it does not allocate memory.

Validation rejects invalid requests: capacities, TTLs, and trend windows
must be finite and positive; model, owner, purpose, and caller labels must be
nonblank; `warm` accepts a positive Ollama duration or `-1` for indefinite
residency. Use `unload()` rather than a zero `keep_alive`.

**Coordination is cooperative.** Direct Ollama calls can bypass it. `force=True`
on `unload`, `ensure_free`, or `warm` overrides claims, busy protection, or
reservation admission; use it only after resolving competing work. It does not
bypass an unreadable coordination ledger or another pending mutation.
`busy=null` means unknown and does not block eviction on its own. See
[coordination details](docs/coordination.md).

## Understand the readings

| `pressure.state` | Meaning |
| --- | --- |
| `ok` | Nothing is wrong, and every input that claim rests on was actually read. Missing evidence yields `unknown`, never `ok`. |
| `tight` | Less than 1 GiB is free on the selected GPU. |
| `degraded` | Ollama placed part of a model on the CPU; expect slower inference. |
| `thrashing` | The driver is paging VRAM: in at least 5 of the last 10 seconds, WDDM's memory manager kept the GPU busy. |

Missing required telemetry produces `pressure.state="unknown"`. Read
`pressure.coverage` and top-level `observations` to see which evidence was
available. Paging detection is best effort. It needs a paging reading for the
selected GPU, which vram-mcp takes on Windows from a sampler that starts with
the server. On Linux no such reading exists, so `pressure` reads `unknown`
where it would read `ok`. The detection is a measured correlation, not a WDDM
contract; see [the acceptance notes](docs/acceptance.md#paging-non-local-memory-and-why-linux-cannot-read-ok).
Non-local memory is reported but does not decide the verdict. CPU
offload is tracked separately. Disabling auditing
stops history detection and trend sampling, but does not disable current GPU or
process observations.

Use `history()` to investigate a model disappearing and `trend(hours=1)` to
review memory changes. Trends are sampled when `vram_status()` reads the GPU;
there is no background sampler. See [diagnostics](docs/coordination.md#diagnostics).

## Tools

The client's MCP schema provides full arguments and defaults.

| Tool | Purpose |
| --- | --- |
| `vram_status` | Inspect GPU memory, models, claims, activity, and pressure. |
| `list_loaded` | List resident models and their claim/busy details. |
| `ensure_free` | Try to reach a free-memory target by unloading unprotected models. |
| `unload` | Evict a named model, respecting claims and recent activity. |
| `warm` | Load a model for a chosen `keep_alive`, checking reservations. |
| `claim`, `reserve` | Declare model use or capacity needed for other GPU work. |
| `renew`, `release` | Extend or end your claim or reservation. |
| `list_claims` | See active model claims and capacity reservations. |
| `history`, `trend` | Review recorded actions, observed changes, and memory samples. |
| `advise` | Get configuration suggestions; no settings are changed. |

## Guides

| Guide | Read it for |
| --- | --- |
| [Configuration](docs/configuration.md) | Source installation, environment variables, and connection troubleshooting. |
| [Coordination and diagnostics](docs/coordination.md) | Claims, reservations, protection limits, pressure, and audit history. |
| [Hardware acceptance](docs/acceptance.md) | The opt-in run against a real GPU and Ollama, and the environments it has validated. |

## Development

```sh
git clone https://github.com/sushiHex/vram-mcp.git
cd vram-mcp
python -m venv .venv
```

Activate with `source .venv/bin/activate` on Linux or
`.\.venv\Scripts\Activate.ps1` in PowerShell, then run:

```sh
python -m pip install -e ".[dev]"
python -m pytest -q
```

Tests use mocked GPU, process, and HTTP calls; no GPU or Ollama daemon is needed.
Validation on real hardware is a separate, opt-in run; see
[hardware acceptance](docs/acceptance.md).
CI covers Python 3.10–3.14 on Windows and Linux. Current model management targets
Ollama and NVIDIA; AMD/Intel telemetry and vLLM/llama.cpp management are future
work.

## License

MIT — see [LICENSE](LICENSE). Companion project:
[hardline-mcp](https://github.com/sushiHex/hardline-mcp) for agent messaging.