GPU Manager MCP
Summary: This read-only MCP server helps you discover, monitor, and plan GPU workloads across NVIDIA, AMD, and Intel hardware.
Discover GPUs, hardware/driver info, capabilities, and system-wide summaries.
Monitor live utilization, VRAM, temperature, power, clocks, processes, and full telemetry snapshots.
Query bounded local history for metric trends, peak usage, and health checks.
Estimate whether LLMs fit in VRAM, recommend GPUs for models, find GPUs with free VRAM, compare GPUs, and detect bottlenecks.
Use it safely as a read-only monitoring/planning server: no shell execution, overclocking, fan control, voltage changes, or process termination.
Report unsupported sensors as unavailable with reasons rather than silently zeroing them.
Provides read-only discovery and monitoring of AMD GPUs via amdgpu sysfs/hwmon and rocm-smi fallback, including utilization, VRAM, temperature, power, clocks, and process data where available.
Provides read-only discovery and monitoring of Intel GPUs via i915/xe sysfs and hwmon, covering utilization, temperature, power, and clocks where available; VRAM metrics are reported as unavailable.
Provides read-only discovery and monitoring of NVIDIA GPUs via NVML/pynvml and nvidia-smi fallback, including utilization, VRAM, temperature, power, clocks, and processes where available.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@GPU Manager MCP Will a 7B q4 model with 4k context fit in my GPU's free VRAM?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
GPU Manager MCP
Read-only GPU monitoring and AI-workload planning over the Model Context Protocol. One stdio server, any MCP client, no LLM required.
Discover NVIDIA, AMD and Intel GPUs, watch utilization / VRAM / temperature / power / clocks, keep bounded local history in SQLite, and estimate whether your LLM actually fits before you load it.
Get started in 60 seconds
git clone https://github.com/quxlzx/gpu-manager-mcp
cd gpu-manager-mcp
python -m venv .venv && .venv/bin/pip install -e .
.venv/bin/gpu-manager-mcp summary # sanity check: see your GPUs without any MCP clientPoint any MCP client at it (from examples/mcp-client-config.json):
{
"mcpServers": {
"gpu-manager": {
"command": "/path/to/gpu-manager-mcp/.venv/bin/gpu-manager-mcp",
"args": ["stdio"]
}
}
}Feedback welcome — if a sensor reads wrong on your hardware, or a tool is confusing, open an issue; validated hardware gets recorded in docs/hardware-validation.md.
Cross-platform backends — NVML (via
pynvml) andnvidia-smifor NVIDIA; amdgpu sysfs (no ROCm needed) androcm-smifallback for AMD; i915/xe sysfs for Intel.Honest telemetry — every reading carries a timestamp, unit and source. Unsupported sensors are reported as
available: falsewith a reason, never silently zeroed.Read-only by design — no shell tool, no overclocking, no voltage, no fan control, no process termination. External helpers run from a fixed allowlist with timeouts and no shell interpolation.
AI-aware —
estimate_model_fit()computes weights + runtime overhead + KV-cache needs for a given quantization, context length and parallel request count, then checks it against live free VRAM.
Related MCP server: MCP System Info Server
Installation
pip install gpu-manager-mcp # (once published; see below)
# or from source
git clone https://github.com/quxlzx/gpu-manager-mcp
cd gpu-manager-mcp
python -m venv .venv && .venv/bin/pip install -e .Optional NVIDIA NVML bindings (recommended on NVIDIA hosts):
pip install "gpu-manager-mcp[nvidia]" # or: pip install nvidia-ml-pyRequirements: Python 3.11+. Linux is fully supported; on Windows the AMD backend is not yet implemented (see Known limitations).
Supported GPUs
Vendor | Path | Util | VRAM | Temp | Power | Clocks | Processes |
NVIDIA (GeForce/Quadro/RTX/Tesla) | NVML ( | ✅ | ✅ | ✅ | ✅* | ✅* | ✅* |
NVIDIA (fallback) |
| ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
AMD (Linux, amdgpu driver) | sysfs + hwmon | ✅* | ✅ | ✅* | ✅* | ✅* | via |
Intel (Linux, i915/xe) | sysfs + hwmon | ✅* | ❌ | ✅* | ✅* | ✅* | ❌ |
✅* = depends on card/driver; unavailable sensors are reported with the
driver's own error, not guessed.
Connecting an MCP client
Generic stdio configuration (any MCP client):
{
"mcpServers": {
"gpu-manager": {
"command": "/path/to/.venv/bin/gpu-manager-mcp",
"args": ["stdio"]
}
}
}Hermes Agent (~/.hermes/config.yaml):
mcp_servers:
gpu-manager:
command: /path/to/.venv/bin/gpu-manager-mcp
args: ["stdio"]
timeout: 60Claude Desktop / Cursor / Cline: same command + args shape as above.
Tools
Discovery: list_gpus, get_gpu_info, get_system_gpu_summary
Monitoring: get_gpu_utilization, get_gpu_memory, get_gpu_temperature,
get_gpu_power, get_gpu_clocks, get_gpu_processes,
get_all_gpu_metrics
History: get_gpu_history, get_gpu_peak_usage, get_gpu_health
AI planning: estimate_model_fit, recommend_gpu_for_model,
find_available_gpu, compare_gpus, detect_gpu_bottleneck
Always call list_gpus() first — every other tool takes a gpu_id from it.
Example
estimate_model_fit(gpu_id="amd-0000:03:00.0", model_size_b=7, quantization="q4", context_length=4096, parallel_requests=1, architecture="qwen2-7b") — real output from an 8 GiB RX 6600 XT:
{
"gpu_id": "amd-0000:03:00.0",
"available": true,
"weights_gib": 3.26,
"runtime_overhead_gib": 0.5,
"kv_cache_gib_per_request": 0.22,
"total_gib": 3.98,
"vram_total_gib": 7.98,
"vram_free_gib": 7.97,
"fits": true,
"headroom_gib": 3.99,
"assumptions": ["runtime overhead = max(0.5 GiB, 5% of weights)",
"KV cache from architecture 'qwen2-7b'"],
"warnings": []
}A metric that the hardware cannot provide looks like:
{
"available": false,
"reason": "sensor busy — the GPU power management interface refused the read (common when the card is in a low-power state)",
"source": "amd-sysfs"
}Configuration
Environment variables (all optional):
Variable | Default | Meaning |
|
| background sampling period (s) |
|
| history window kept |
|
| hard row cap (bounded storage) |
|
| SQLite location |
|
| telemetry cache TTL (s) for <250 ms repeat queries |
|
| timeout for allowlisted helper CLIs |
|
| enable Streamable HTTP transport |
|
| loopback-only HTTP bind |
Security guidance
The server exposes monitoring only. There is no tool that executes arbitrary commands, changes clocks/fans/voltages, or kills processes.
Helper CLIs (
nvidia-smi,rocm-smi,intel_gpu_top) are invoked from a fixed allowlist with argument arrays and timeouts — never via a shell and never with user-supplied strings.gpu_idvalues are validated against discovered devices; unknown or crafted ids are rejected with a clean error.The optional HTTP transport binds to loopback by default and refuses non-loopback hosts unless you explicitly change it — and you should not, without adding authentication in front of it. Do not expose this server to the public internet.
History is local SQLite with a bounded row cap; nothing leaves the machine.
Known limitations
Windows: AMD telemetry via sysfs is Linux-only; a Windows backend is planned (AMD-ADL / PDH or
amdsmibindings). The server runs on Windows but will report no AMD devices.AMD hwmon reads can return EBUSY when the card is in a low-power state; the reading is reported as unavailable with that explanation.
Intel VRAM is not exposed by i915/xe sysfs; memory metrics are reported unavailable rather than estimated.
KV-cache estimates are architecture-dependent. Without an
architecturehint the server uses documented default assumptions and flags them. Treat all fit numbers as engineering estimates.Per-process VRAM requires NVML or
rocm-smi; plain sysfs cannot provide it.
Troubleshooting
No GPUs found on Linux — check
/sys/class/drm/card*/device/vendorexists and the driver isamdgpu/i915/xe; NVIDIA needs the proprietary driver +nvidia-ml-pyornvidia-smion PATH.sensor busyon AMD — normal at idle on some cards; retry under load or checkpower_statein sysfs.Permission denied — NVML on some setups needs root or
CAP_SYS_ADMIN; sysfs sensors are world-readable on standard distros.History empty — the sampler runs inside the server process; give it a few sample intervals, or call
get_all_gpu_metrics()to force samples.
Development
pip install -e ".[dev]"
pytest # unit + MCP client tests (mocked hardware)
ruff check src tests
mypy src
python -m build # package validationLicense
MIT — see LICENSE.
Available Tools
18 toolscompare_gpusB
Side-by-side comparison of specs and live telemetry for several GPUs.
Inputs: gpu_ids (required).
detect_gpu_bottleneckB
Heuristic bottleneck analysis (compute/thermal/power/memory) from sampled history. Reports when evidence is insufficient.
Inputs: gpu_id (required), window_seconds.
estimate_model_fitB
Estimate whether an LLM fits in one GPU's VRAM. Rough estimate: weights + runtime overhead + KV cache. Pass 'architecture' (e.g. llama-3-8b, mistral-7b) for an architecture-aware KV cache; without it, default GQA assumptions are used and flagged.
Inputs: gpu_id (required), model_size_b (required), architecture, quantization, +2 more.
find_available_gpuA
GPUs with at least required_vram_gb (GiB) free right now.
Inputs: required_vram_gb (required).
get_all_gpu_metricsB
Full telemetry snapshot for every GPU at once.
Inputs: no named parameters.
get_gpu_clocksC
Current graphics clock in MHz.
Inputs: gpu_id (required).
get_gpu_healthB
Health check: thermal/memory/power issues from live + historical data.
Inputs: gpu_id (required).
get_gpu_historyB
Historical samples for one metric over a time window. Metrics: utilization_percent, memory_used_bytes, memory_total_bytes, memory_free_bytes, temperature_celsius, power_watts, clock_mhz.
Inputs: gpu_id (required), metric (required), duration_seconds.
get_gpu_infoC
Hardware and driver information for one GPU.
Inputs: gpu_id (required).
get_gpu_memoryC
VRAM total/used/free in bytes for one GPU.
Inputs: gpu_id (required).
get_gpu_peak_usageC
Max/min/avg of a metric over a window.
Inputs: gpu_id (required), metric, duration_seconds.
get_gpu_powerC
GPU package power draw in watts.
Inputs: gpu_id (required).
get_gpu_processesB
Processes using the GPU with per-process VRAM, when the driver exposes it (NVML or rocm-smi). Returns supported=false otherwise.
Inputs: gpu_id (required).
get_gpu_temperatureB
GPU temperature in Celsius (edge sensor where available).
Inputs: gpu_id (required).
get_gpu_utilizationC
GPU core utilization percentage for one GPU.
Inputs: gpu_id (required).
get_system_gpu_summaryB
System-wide overview: every GPU with one-line key metrics.
Inputs: no named parameters.
list_gpusA
Discover all GPUs (NVIDIA/AMD/Intel). Returns stable gpu_ids, names, VRAM totals, PCI bus ids, driver versions and supported telemetry capabilities per device.
Inputs: no named parameters.
recommend_gpu_for_modelC
Rank all discovered GPUs by estimated fit for the given model.
Inputs: model_size_b (required), architecture, quantization, context_length, +1 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 18 tools
The per-metric tools (utilization, memory, temperature, power, clocks) are clearly scoped to one signal each, and the aggregate tools (get_all_gpu_metrics, get_system_gpu_summary, compare_gpus) are distinguishable by their descriptions. Minor overlap remains between list_gpus, get_gpu_info, and get_system_gpu_summary, which all return device-level inventory/summary data that could confuse selection.
All 18 tools use snake_case verb_noun phrasing (get_gpu_memory, get_gpu_history, estimate_model_fit, recommend_gpu_for_model, find_available_gpu, detect_gpu_bottleneck). The get_gpu_* telemetry family is highly predictable, and the analysis tools follow the same verb-first convention.
18 tools is on the heavy side for a GPU monitoring server, and the narrow per-metric getters largely duplicate what get_all_gpu_metrics returns. Still, each tool has a single defensible purpose and the surface is far from bloated given the breadth of discovery, telemetry, history, and advisory operations.
Coverage is strong: discovery, per-metric and bulk telemetry, historical samples, health checks, model-fit estimation, and bottleneck analysis form a coherent lifecycle. Gaps are modest — no fan speed, ECC/memory-error counters, MIG/partition details, or any configuration/mutation operations, but agents can work around these for read-only monitoring.
Maintenance
Related MCP Connectors
Persistent memory, knowledge graph, task management and more for AI agents via MCP.
Massed Compute MCP — GPU inventory, VM lifecycle, billing, SSH keys, and setup recipes.
GPU compute-asset identity registry: resolve device identity, verify history, buy official reports.
Live GPU rental market: 2,500+ offers across a dozen provider feeds. History, watches, limit orders.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables real-time monitoring of system resources including CPU, GPU (NVIDIA, Apple Silicon, AMD/Intel), memory, disk, network, and processes across Windows, macOS, and Linux platforms through natural language queries.3MIT
- FlicenseBqualityDmaintenanceA lightweight MCP server that provides real-time hardware statistics including CPU, memory, disk, and NVIDIA GPU usage. It enables users to monitor system performance and retrieve comprehensive host machine specifications through a standardized interface.1-
- AlicenseAqualityBmaintenanceExposes NVIDIA GPU metrics (info, utilization, VRAM, temperature) via MCP tools for real-time querying from AI assistants.5MIT
- AlicenseNot gradedqualityBmaintenanceAn MCP server that enables AI agents to query real-time NVIDIA GPU metrics like utilization, memory, temperature, and power without external monitoring tools.15Apache 2.0