Inference AIops
OfficialGoverned AI-ops server for GPU inference clusters: observe/RCA vLLM/SGLang/TGI metrics, inspect Ray Serve/cluster/jobs/GPU state, and perform guarded vLLM/Ray Serve writes with audit, risk tiers, budgets, and undo tokens.
Read vLLM metrics and RCA: TTFT/TPOT/e2e latency, queue depth, KV-cache stats, prefix-cache locality; diagnose latency spikes and low utilization.
Engine-agnostic observability for vLLM/SGLang/TGI: health, inventory/running models, request metrics, queue depth, latency RCA.
Inspect Ray Serve and cluster: deployments, replicas, autoscale config, cluster resources, dashboard status, jobs, GPU utilization.
Manage vLLM models/LoRA/Sleep Mode: list/info models, load/unload LoRA, sleep/wake where dev-mode routes exist.
Perform guarded writes: scale up/down/to zero, update autoscale, deploy/undeploy/redeploy applications, cancel Ray jobs, and routing-policy updates where REST-supported.
Use governance harness: every call audited, risk-tier labels, token/call budget guard, undo-list/undo-apply for reversible writes, and dry_run previews.
Estimate cost: derive $/1M-token unit cost from live throughput and GPU hourly cost.
Provides tools for managing Ray Serve deployments, replicas, and Ray Jobs, including scaling, autoscaling, draining, and monitoring of GPU inference clusters.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Inference AIops diagnose the latency spike in the production deployment"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Inference AIops
Audit availability: Real writes require a committed
pendingaudit row and stop before contacting the platform if storage is unavailable. Reads and previews may continue with a warning; new audit rows store outcome summaries rather than full responses.
Disclaimer: Community-maintained open-source project. Not affiliated with, endorsed by, or sponsored by the vLLM or Ray projects or any inference-serving vendor. Product and trademark names belong to their owners. MIT licensed.
Governed AI-ops for GPU inference clusters — vLLM (OpenAI API + Prometheus
/metrics) and Ray Serve / Ray Jobs (Ray dashboard), plus the single-process
serving engines SGLang and TGI (Text Generation Inference) — with a
built-in governance harness: unified audit log, policy engine, token/runaway
budget guard, undo-token recording, and descriptive risk-tier labels on every
audit row. It parses each engine's Prometheus /metrics directly (no Prometheus
server required) and
probes the Ray dashboard independently. A bearer token is optional (many
stacks run open).
Serving engines. vLLM is the flagship (full Ray Serve control plane: scale, drain, autoscale, LoRA, hot-swap). SGLang and TGI are supported for engine-agnostic observability — health, running-model identity, request-latency metrics, queue depth, and latency RCA — read from each engine's own endpoints and metric names. Being single-process servers, they have no Ray-shaped scale/drain API: those writes return a teaching error pointing you at a real horizontal-scale layer (Ray Serve / Kubernetes / a load balancer).
What it does
The flagship value is root-cause analysis, wrapped in guarded reads and writes:
diagnose_latency_spike(flagship RCA) — when TTFT/TPOT/e2e latency climbs, it correlates queue depth (running vs waiting), KV-cache pressure / preemptions, and prefix-cache locality into a ranked cause plus the specific knob to turn (add replicas, raisemax-num-seqs, fix routing, enlarge KV cache). Every flag is a number, not a black-box verdict.diagnose_low_utilization— the inverse: idle GPUs, over-provisioned replicas, or routing that strands a cache-warm replica → what to scale down.Prometheus-native — reads vLLM's
/metricsendpoint directly; no Prometheus/Grafana deployment needed.Governance-grade — the first governance-grade entrant in this niche: audit + budget + risk-tier approval + undo-token + prompt-injection sanitize, with dry-run + double-confirm on the fragile prod ops (scale-down, scale-to-zero, drain, redeploy, hot-swap) the community reports as dangerous.
Laptop self-test — ~80% of the tool self-tests free: vLLM on a single GPU or CPU-mock + Ray in one local container (
ray start --head).
Related MCP server: vibops-mcp
What this tool does, and does not, decide
It delivers inference-cluster operations — reads and writes — accurately and efficiently, and records every one of them. It does not decide whether a write is allowed to happen. That is the agent's judgement, or the permission of the environment you connect it with: restrict the network path so it can only reach the read/metrics endpoints, or run the Ray dashboard without its job-submission API, and the writes fail at the server — the place that actually owns the permission.
So there is no read-only switch, no policy file, no approval gate to configure.
The one thing the tool guarantees is that nothing is silent: every call, over
MCP and over the CLI alike, lands an audit row in
~/.inference-aiops/audit.db, and destructive writes still capture their
before-state and record an inverse where one exists.
Each tool declares a
risk_level, kept in agreement with its[READ]/[WRITE]documentation tag by a test, and carried into the audit row as a descriptive tier — so a reviewer can see at a glance that a row was a high-risk scale-to-zero. It is a label, not a gate.
Running a smaller / local model? See agent-guardrails.md — it lists the guardrails this tool enforces for you (so you don't spend prompt budget restating them) and gives a ready-made system prompt for what's left.
Capability matrix (39 MCP tools)
Group | Tools | Count | R/W (risk) |
Metrics & RCA (vLLM) |
| 5 | read |
Engine-agnostic (vLLM / SGLang / TGI) |
| 5 | read |
Ray Serve (read) |
| 4 | read |
Ray Serve (write) |
| 5 | write (med / high) |
Models / vLLM |
| 5 | read + write (med) |
Sleep Mode / vLLM (needs |
| 2 | write (high / med) |
Ray cluster / jobs / GPU |
| 6 | read + write (med / high) |
Deploy lifecycle |
| 4 | write (med / high) |
Cost |
| 1 | read |
The engine-agnostic group works against any supported engine (including vLLM); use it for SGLang/TGI targets or a uniform view across a mixed fleet. The Ray Serve / cluster / deploy write groups are vLLM-only (Ray control plane) — they teach-and-refuse on a SGLang/TGI target.
23 read, 16 write. High-risk writes (scale_replicas_down,
scale_to_zero, drain_replica, lora_unload, model_sleep,
replica_restart, model_undeploy, deployment_redeploy) all support
dry_run + double-confirm; reversible writes record an undo descriptor.
Sleep Mode requires a dev-mode server. vLLM registers
/sleep,/wake_upand/is_sleepingonly when started withVLLM_SERVER_DEV_MODE=1. Against any other server these three tools report that the route is absent and why, rather than failing vaguely. Sleep Mode suspends the same model; it does not swap base models — serving a different base model means restarting vLLM with a different--model.
Install
uv tool install inference-aiops # or: pipx install inference-aiopsQuick start
As a Claude Code plugin
One install gives an agent both the skill and the MCP server:
/plugin marketplace add AIops-tools/marketplace
/plugin install inference-aiops@aiops-toolsThe MCP server is fetched with uv and pinned to the
package version this plugin declares, so an audit row can be traced back to the
code that wrote it. Credentials are still configured with inference-aiops init — see below.
As an OpenClaw plugin
The same bundle is published on ClawHub, where one install delivers the skill and its MCP server together:
openclaw plugins install clawhub:@zw008/inference-aiops
openclaw skills info inference-aiops # expect: Visible to model: yesRestart the OpenClaw gateway afterwards so it loads the plugin. The MCP server is
fetched with uv, pinned to this exact release, so
uvx has to be on PATH — without it the skill still installs but reports
Visible to model: no. Credentials are configured exactly as below.
As a CLI or standalone MCP server
inference-aiops init # wizard: engine (vllm/sglang/tgi) + host + port + scheme
inference-aiops doctor # vLLM: probes Ray + vLLM; SGLang/TGI: engine health + inventory
inference-aiops overview # deployments + total replicas + queue backpressure
inference-aiops metrics diagnose # why is inference slow? ranked RCA + the knob to turn
inference-aiops serve list # Ray Serve deployments + replica countsRun as an MCP server (stdio) for the full 39-tool surface:
export INFERENCE_AIOPS_MASTER_PASSWORD=... # only if a bearer token is stored
inference-aiops mcpWhere that password then lives: an exported variable is readable by every process this shell starts and is recorded by shell history. On a shared or long-lived host, prefer the interactive prompt, or inject it from a secret manager for the life of the one command that needs it.
The CLI is a convenience subset (init, overview, serve …, metrics …,
secret …, doctor, mcp); the full 39 tools are exposed via the MCP server.
Governance
Every MCP tool passes through the bundled @governed_tool harness. It does not
decide whether a write is permitted — see What this tool does, and does not,
decide above — but it records every call:
Audit — every call (params, outcome summary, status, duration, risk tier, and any approver/rationale annotation) logged to
~/.inference-aiops/audit.db(relocatable viaINFERENCE_AIOPS_HOME).Budget / runaway guard — a safety backstop, not authorization: token and call budgets trip a circuit breaker on tight poll/retry loops.
Risk tier — each audit row carries a descriptive tier derived from the tool's
risk_level; it is a label, not a gate.INFERENCE_AUDIT_APPROVED_BY/INFERENCE_AUDIT_RATIONALEare optional annotations recorded when set, never required.Undo recording — reversible writes (scale, autoscale-config, routing, hot-swap, LoRA load) record an inverse descriptor.
Supported scope + limitations
Behaviour is exercised by the test suite against mocked vLLM /metrics, vLLM
OpenAI API, and Ray dashboard responses. ~80% of the tool self-tests on a
laptop — vLLM on a single GPU or CPU-mock plus a local one-node Ray head. It
has not been run against a live production cluster; see
docs/VERIFICATION.md for the live-verification
checklist.
Unverified against real hardware / topology:
multi-GPU tensor-parallel / pipeline-parallel deployments,
real GPU thermal / throttle telemetry (utilisation is best-effort from the Ray dashboard's
/api/nodes),multi-node drain and node-reboot orchestration.
The fastest live check is inference-aiops doctor; the full checklist lives in
docs/VERIFICATION.md.
Missing a capability?
This is the GPU-inference member of the AIops-tools family (governed AI-ops with audit + budget + undo + risk tiers). If a vLLM or Ray capability you need is missing, or your stack speaks a dialect these tools don't yet handle — open an issue or a PR. Contributions welcome.
Available Tools
39 toolsautoscale_config_getARead-only
[READ] Autoscale bounds (min/max replicas, target ongoing requests).
Args: application: Serve application name. deployment: Deployment name. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target.
autoscale_config_updateADestructive
[WRITE][risk=medium] Live-tune autoscale bounds without a redeploy (reversible).
Args: application: Serve application name. deployment: Deployment name. min_replicas: New floor (omit to leave unchanged). max_replicas: New ceiling (omit to leave unchanged). target_ongoing_requests: New per-replica concurrency target (omit to leave). target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target, max_replicas, +2 more.
cost_per_tokenARead-only
[READ] Attribute a $/1M-token unit cost from live vLLM throughput.
Multiplies the current generation-throughput gauge by the supplied GPU
hourly cost to derive the cost of serving 1M tokens; degrades to an
insufficient-data forecast when no throughput metric is present.
Args: gpu_hourly_cost: Hourly cost of a single GPU (e.g. cloud on-demand rate). num_gpus: Number of GPUs backing the deployment; defaults to 1. target: Inference target name from config; omit for the default.
Inputs: gpu_hourly_cost (required), target, num_gpus.
deployment_redeployADestructive
[WRITE][risk=high] Force a deployment to re-apply new config.
Applies the new config immediately and can drop unfinished requests — pass dry_run=True to preview.
Args: application: Serve application name. deployment: Deployment name within the application. dry_run: If True, preview without redeploying. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target, dry_run.
deployment_statusARead-only
[READ] One deployment's status + current/target replica count.
Args: application: Serve application name (from serve_deployment_list). deployment: Deployment name within the application. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target.
diagnose_engine_latencyARead-only
[READ][RCA] Rank the probable cause of a latency spike for any serving engine.
Correlates whichever signals the engine exposes (queue backpressure, KV/token-cache pressure, cache locality) into a ranked cause + the knob to turn — the engine-agnostic counterpart to diagnose_latency_spike (vLLM).
Args: target: Inference target name from config; omit for the default.
Inputs: target.
diagnose_latency_spikeARead-only
[READ][RCA] Rank the probable cause of a TTFT/latency spike + the knob to turn.
Correlates queue depth, KV-cache pressure/preemption, and prefix-cache locality into a ranked cause list — call this first on "why is inference slow".
Args: target: Inference target name from config; omit for the default.
Inputs: target.
diagnose_low_utilizationARead-only
[READ][RCA] Explain an under-used GPU (batching / idle / overprovision).
Args: target: Inference target name from config; omit for the default.
Inputs: target.
drain_replicaADestructive
[WRITE][risk=high] Drain one replica — NOT available over Ray's REST API.
Ray Serve exposes no per-replica drain endpoint (only the whole-cluster declarative config); individual-replica draining is a Python-API capability only. This tool refuses with a teaching error. To retire surplus replicas, scale the deployment down — the controller drains them gracefully. The dry_run preview reports the same unavailability rather than a false green.
Args: application: Serve application name. deployment: Deployment name. replica_id: Replica id (from replica_list). dry_run: If True, report availability without attempting a drain. target: Inference target name from config; omit for the default.
Inputs: deployment (required), replica_id (required), application (required), target, +1 more.
engine_healthARead-only
[READ] Liveness of the serving engine (vLLM / SGLang / TGI) via its health probe.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
engine_inventoryARead-only
[READ] Running-model identity + engine server info (engine-agnostic).
vLLM / SGLang report served ids from /v1/models; TGI's single model id comes from /info.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
engine_queue_depthARead-only
[READ] Running vs waiting requests — the leading backpressure signal (any engine).
Args: target: Inference target name from config; omit for the default.
Inputs: target.
engine_request_metricsBRead-only
[READ] TTFT / TPOT / e2e latency + generation-token totals (where the engine exposes them).
Args: target: Inference target name from config; omit for the default.
Inputs: target.
gpu_utilizationARead-only
[READ] Per-node GPU count, utilisation %, and memory used/total.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
kv_cache_statsBRead-only
[READ] KV-cache utilisation, prefix-cache hit rate, and preemption count.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
lora_loadADestructive
[WRITE][risk=medium] Hot-load a LoRA adapter onto the running engine.
Args: lora_name: Adapter name to register. lora_path: Local path or HF repo id of the adapter weights. target: Inference target name from config; omit for the default.
Inputs: lora_name (required), lora_path (required), target.
lora_unloadADestructive
[WRITE][risk=high] Hot-unload a LoRA adapter (traffic on it starts failing).
Pass dry_run=True to preview.
Args: lora_name: Adapter name to unload (from model_list). dry_run: If True, preview without unloading. target: Inference target name from config; omit for the default.
Inputs: lora_name (required), target, dry_run.
model_deployADestructive
[WRITE][risk=medium] Deploy a Serve application from an import path.
The app is merged into the cluster's declarative Serve config (existing apps preserved). Replica counts come from the app's own config; set them afterward with serve scale (deployment names are known only once the app materialises).
Args: application: Serve application name to create/replace. import_path: Python import path of the Serve app (e.g. 'module:app'). target: Inference target name from config; omit for the default.
Inputs: application (required), import_path (required), target.
model_infoARead-only
[READ] One model's config: max length, root/parent, permission (best-effort).
Args: model_id: Model id (from model_list). target: Inference target name from config; omit for the default.
Inputs: model_id (required), target.
model_is_sleepingARead-only
[READ] Whether the vLLM engine is suspended in Sleep Mode.
Returns isSleeping: true (suspended, serving nothing), false (serving), or null when the engine did not report it — null means UNKNOWN, not awake.
Sleep Mode exists only on servers started with VLLM_SERVER_DEV_MODE=1; on any other server this reports that the route is absent rather than failing vaguely.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
model_listARead-only
[READ] All served vLLM models, LoRA adapters flagged.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
model_sleepADestructive
[WRITE][risk=high] Suspend the engine via Sleep Mode (it stops serving requests).
Frees GPU memory between bursts. level=1 offloads the weights to CPU RAM and wakes fast; level=2 discards them, so waking reloads from disk. The engine serves nothing until model_wake. Pass dry_run=True to preview.
Sleep Mode exists only on servers started with VLLM_SERVER_DEV_MODE=1; on any other server this reports that the route is absent rather than failing vaguely.
Args: level: 1 to offload weights to CPU RAM, 2 to discard them (default 1). dry_run: If True, preview without suspending. target: Inference target name from config; omit for the default.
Inputs: level, target, dry_run.
model_undeployADestructive
[WRITE][risk=high] Tear down a whole Serve application (removes all deployments).
Irreversible without the original import path — pass dry_run=True to preview.
Args: application: Serve application name (from serve_deployment_list). dry_run: If True, preview without undeploying. target: Inference target name from config; omit for the default.
Inputs: application (required), target, dry_run.
model_wakeADestructive
[WRITE][risk=medium] Resume serving after Sleep Mode (weights return to GPU).
The inverse of model_sleep. Pass dry_run=True to preview. Records no undo: vLLM never reports which sleep level the engine was at, so re-sleeping would have to guess between level 1 and level 2 — call model_sleep with the level you want instead.
Sleep Mode exists only on servers started with VLLM_SERVER_DEV_MODE=1; on any other server this reports that the route is absent rather than failing vaguely.
Args: dry_run: If True, preview without waking. target: Inference target name from config; omit for the default.
Inputs: target, dry_run.
queue_depthBRead-only
[READ] Running vs waiting requests — the leading backpressure signal.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
ray_cluster_resourcesBRead-only
[READ] Cluster-wide CPU/GPU capacity + headroom + pending placement groups.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
ray_dashboard_statusBRead-only
[READ] Serve controller health + app/deployment counts.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
ray_job_cancelADestructive
[WRITE][risk=medium] Stop a submitted/running Ray job.
Args: job_id: Job id (from ray_job_list). target: Inference target name from config; omit for the default.
Inputs: job_id (required), target.
ray_job_listARead-only
[READ] Submitted Ray jobs: id, status, entrypoint, start time.
Returns {"jobs": [...], "returned": N, "limit": L, "truncated": bool}.
When truncated is true there are more jobs than were returned — re-run
with a higher limit rather than treating the result as the whole history.
Args: target: Inference target name from config; omit for the default. limit: Maximum job rows to return. Default 100.
Inputs: limit, target.
replica_listBRead-only
[READ] Replica states for one deployment (running/starting/draining).
Args: application: Serve application name. deployment: Deployment name. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target.
replica_restartADestructive
[WRITE][risk=high] Restart one Serve replica — NOT available over Ray's REST API.
Ray Serve exposes no per-replica restart endpoint over REST; the controller respawns unhealthy replicas on its own. This tool refuses with a teaching error. To force a cycle, scale the deployment down then up, or redeploy. The dry_run preview reports the same unavailability rather than a false green.
Args: application: Serve application name. deployment: Deployment name. replica_id: Replica id (from replica_list). dry_run: If True, report availability without attempting a restart. target: Inference target name from config; omit for the default.
Inputs: deployment (required), replica_id (required), application (required), target, +1 more.
request_metricsBRead-only
[READ] vLLM TTFT / TPOT / e2e latency + generation-token totals.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
routing_policy_updateADestructive
[WRITE][risk=medium] Switch a deployment's request-routing policy.
NOTE: Ray Serve's request-routing policy is a code-level option on the @serve.deployment decorator, NOT a field settable over the Dashboard REST API — there is no REST endpoint for it. This tool therefore refuses with a teaching error rather than pretend; change the policy in the deployment's source and redeploy.
Args: application: Serve application name. deployment: Deployment name. policy: Desired routing policy (prefix_aware / round_robin / session_affinity). target: Inference target name from config; omit for the default.
Inputs: policy (required), deployment (required), application (required), target.
scale_replicas_downADestructive
[WRITE][risk=high] Lower a deployment's replica count on prod (reversible → prior).
Fewer replicas can strand in-flight requests — pass dry_run=True to preview.
Args: application: Serve application name. deployment: Deployment name. num_replicas: New (lower) replica count. dry_run: If True, preview without scaling. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), num_replicas (required), target, +1 more.
scale_replicas_upADestructive
[WRITE][risk=medium] Raise a deployment's replica count (reversible → prior).
Pass dry_run=True to preview: it reads the deployment's current count, so the preview reports the real from→to rather than only the requested target — and it fails here, not mid-write, if the deployment does not exist.
Args: application: Serve application name. deployment: Deployment name. num_replicas: New (higher) replica count. dry_run: If True, preview without scaling. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), num_replicas (required), target, +1 more.
scale_to_zeroADestructive
[WRITE][risk=high] Park a deployment at 0 replicas (reversible → prior count).
Stops the cost bleed but adds cold-start latency and can strand the ingress — pass dry_run=True to preview.
Args: application: Serve application name. deployment: Deployment name. dry_run: If True, preview without scaling to zero. target: Inference target name from config; omit for the default.
Inputs: deployment (required), application (required), target, dry_run.
serve_deployment_listBRead-only
[READ] All Ray Serve deployments: status, replica count, target.
Args: target: Inference target name from config; omit for the default.
Inputs: target.
undo_applyADestructive
[WRITE][risk=medium] Apply a recorded undo by dispatching its inverse tool.
The inverse runs through its own governed tool, so it is audited under its own name and risk level. Pass dry_run=True to preview the inverse call without executing it. A token can only be applied once.
Args:
undo_id: The undoId from undo_list (or an _undo_id in a write result).
dry_run: If True, preview the inverse tool + params without running it.
target: Passed through to the inverse tool when it accepts a target.
Inputs: undo_id (required), target, dry_run.
undo_listARead-only
[READ] List recorded, not-yet-applied undo tokens (most recent first).
Each entry names the original tool, the inverse tool that undo_apply
would run, and a human note. Use the undoId with undo_apply.
Returns {"undos": [...], "returned": N, "limit": L, "truncated": bool}.
truncated is measured (one extra row is fetched), not guessed from a
length coincidence: when it is true there are MORE tokens than shown, so
re-run with a higher limit rather than reporting the list as complete.
Each entry carries effectVerified. False means the original write lost
its response, so the change it reverses is PROBABLE, not confirmed — check
the live state before applying, and do not report the result as a restore
of a state that may never have been reached.
Args: limit: Max rows to return (default 50, capped at 500). target: Unused (undo state is host-local); accepted for CLI uniformity.
Inputs: limit, target.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 39 tools
Several tool pairs overlap heavily: engine_request_metrics vs request_metrics, engine_queue_depth vs queue_depth, and diagnose_engine_latency vs diagnose_latency_spike differ only by the engine-agnostic vs vLLM-specific nuance, which relies on careful reading. model_undeploy, scale_to_zero, and deployment_redeploy also have fuzzy boundaries when the goal is 'stop serving'. Descriptions do help disambiguate, but the near-duplicate pairs invite misselection.
The set mixes verb_noun (scale_replicas_up, lora_load, model_deploy) with noun_verb patterns (engine_request_metrics, serve_deployment_list, replica_restart, deployment_status, cost_per_token). Prefixes like engine_/ray_/serve_ add grouping but are applied inconsistently. It stays readable but no single predictable convention governs the set.
39 tools is well above the comfortable range and the surface is heavy for what is essentially a monitoring/ops console. The count is inflated by engine-agnostic vs engine-specific duplicates and by three tools that only ever return a refusal. It feels overstated for the domain.
Coverage is broad: deploy/undeploy, replica scaling, autoscale config, LoRA load/unload, sleep/wake, health/inventory/queue/cache metrics, RCA diagnostics, Ray cluster/job control, and an undo lifecycle. Gaps are minor, though routing_policy_update, replica_restart, and drain_replica exist only to reject requests, which adds surface without capability.
Maintenance
Related MCP Connectors
Autonomous and governed SRE & MLOps control plane for AI agent platforms.
Enterprise AI Control Plane: governance, guardrails, spend tracking, compliance & smart routing.
Real-time infrastructure monitoring with metrics, logs, alerts, and ML-based anomaly detection.
AI agent observability for production traces, natural-language insights, and improvement loops.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceProvides AI assistants with direct access to Red Hat OpenShift AI observability data, enabling querying of Prometheus metrics, Alertmanager alerts, Loki logs, Grafana dashboards, and Kubernetes cluster state to troubleshoot vLLM inference workloads.5MIT

vibops-mcpofficial
AlicenseBqualityBmaintenanceVibOps MCP is the control plane between your AI agents and your GPU infrastructure. 74 tools covering: GPU fleet management (deploy, scale, monitor across NVIDIA, AMD, Intel, AWS, Google, Groq), Agent Infrastructure Control Plane (per-agent GPU cost, budget enforcement, model policies, dependency graph), governance (AI Act, SOC 2, immutable HMAC audit chain), and GPU FinOps (chargeback, waste..)..117491 PyPI20MIT- AlicenseAqualityBmaintenanceReal-time cluster health monitoring, pre-request NLMS latency prediction, and intelligent prompt routing across multi-instance LLM backends (vLLM, Ollama, SGLang, TGI).43Apache 2.0

kubestellar-mcpofficial
AlicenseNot gradedqualityAmaintenanceEnables AI agents to inspect and operate multiple Kubernetes clusters through natural language, providing multi-cluster diagnostics, RBAC analysis, security checks, and app-centric deployment with GitOps and smart workload placement.2Apache 2.0