Skip to main content
Glama
io.github.sandeepbazar/ocm-mcp-server

io.github.sandeepbazar/ocm-mcp-server

ocm-mcp-server

📖 Read the docs site → ocm-mcp-server.github.io

AgentOps for Kubernetes fleets, done safely.

An MCP server that lets AI agents operate a multi-cluster Kubernetes fleet through an Open Cluster Management hub, with policy, approval, and audit between the model and your clusters.

The agent never holds a kubeconfig. Every write is policy-checked, human-approved, and traced.

License Python MCP OCM Kyverno CI e2e Coverage OpenSSF Scorecard PyPI Release

LinkedIn YouTube

📦 Get it  ·  ✨ Why  ·  🔌 Connect your agent  ·  🧭 Architecture  ·  🧰 Toolsets  ·  🛠️ Tools  ·  💬 Prompts  ·  🔭 Observability  ·  🚀 Quickstart  ·  📖 Wiki  ·  📚 Docs

The whole safe-remediation loop: investigate with free reads, propose a change, get rejected by the guardrails and correct it, wait for a human-signed token, apply, verify, and report from the audit log.


Where to get it, and how it's vetted

  • 📦 PyPI - ocm-mcp-server - pip install ocm-mcp-server (or run directly with uvx ocm-mcp-server). Every release is published straight from CI via OIDC trusted publishing - no long-lived tokens anywhere.

  • 🗂️ Official MCP Registry - listed as io.github.ocm-mcp-server/ocm-mcp-server, so any MCP client or platform that browses the registry can discover and auto-configure this server (package, transport, and required env vars are all in the listing); the registry validates the listing against this repo and the PyPI package.

  • 🐳 Container image on GHCR - docker run ghcr.io/ocm-mcp-server/ocm-mcp-server (kubeconfig mount shown in the deployment guide); built in CI with an SBOM and SLSA provenance attached, vulnerability-gated with Trivy, and signed keyless with Cosign so you can verify what you run.

  • 🛡️ OpenSSF Scorecard - the repo's supply-chain security posture (pinned dependencies, branch protection, signed releases, ...) is scored automatically every week and published for anyone to inspect.

Related MCP server: k8s-aiops

Why this exists

Your team runs many Kubernetes clusters. Sooner or later somebody asks the question: can an AI agent take the 2 a.m. page?

The quickest way to find out is to hand a model kubectl with cluster-admin and watch. In production that experiment ends badly, for three separate reasons:

  • The model is non-deterministic. The same alert can produce a careful diagnosis one run and a kubectl delete the next.

  • The credentials are real. There is no dry run between the model's decision and your production cluster.

  • There is no record. When something breaks, you cannot reconstruct what the agent did, in what order, or on whose authority.

This project starts from a different observation: fleets already have a control point that humans trust every day, the multi-cluster hub. Open Cluster Management (a CNCF project) gives every fleet an inventory (ManagedCluster), a scheduler (Placement), and a delivery channel (ManifestWork). ocm-mcp-server exposes that hub to agents as a small set of typed MCP tools, and puts four independent layers between the model and your clusters:

#

Layer

Enforced by

What it stops

1

Static checks

this server, before anything else

privileged pods, host access, system namespaces, unpinned images, disallowed kinds

2

Policy admission

Kyverno dry-run on the hub

anything your org's policies reject, evaluated inside the ManifestWork envelope

3

Human approval

Ed25519 token signed by ocm-mcp approve on a trusted terminal; the server needs only the public verifier key

any change reaching a cluster without a person consenting to that exact content and operation (one-time token, bound to content + operation + issuer/audience + expiry)

4

Least-privilege RBAC

Kubernetes

everything else; no Secrets, no exec, no deletes outside its own ManifestWorks

None of these layers live in the system prompt, so none of them can be talked out of.

A write is two calls with a person between them. The token is bound to one content hash and one operation, it expires on its own, and offered a second time it is refused.

Two writes, the same four gates. The privileged, unpinned one dies at Layer 1 and never reaches a cluster; the compliant one waits for a person to sign the exact content, then lands and is verified.

A fleet operator's day with Claude, live from a cold start: install from PyPI, claude mcp add, inventory the fleet, reason about placement — then ship a new service the gated way: the privileged :latest shortcut is refused, the pinned proposal is signed by a human, applied with the token, verified, and the whole day is read back from the audit trail. — narrated MP4 · terminal cast.

The same day, driven by four different agents. Identical ten chapters, identical server - only the agent asking changes, which is the whole point of speaking MCP rather than shipping a client. Codex · Gemini (Antigravity CLI). Re-record any of them with hack/demo-record.sh all.

Connect your agent - any MCP client works

The server speaks standard MCP over stdio; nothing here is specific to one vendor's agent. That claim is demonstrated, not asserted: the same ten-chapter operator session is recorded against four different agents - Claude Code, Codex, Gemini through the Antigravity CLI, and IBM Bob Shell - driving the same server against the same fleet, each one really calling the tools, hitting the guardrail refusal, and applying only with a human-signed token. Re-record any of them with hack/demo-record.sh all.

Ready-made configs live in examples/ - see the index for where each file goes:

{
  "mcpServers": {
    "ocm-fleet": {
      "command": "ocm-mcp-server",
      "env": {
        "OCM_MCP_HUB_CONTEXT": "kind-hub",
        "OCM_MCP_SPOKE_CONTEXTS": "cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3"
      }
    }
  }
}
{
  "servers": {
    "ocm-fleet": {
      "type": "stdio",
      "command": "ocm-mcp-server",
      "env": {
        "OCM_MCP_HUB_CONTEXT": "kind-hub",
        "OCM_MCP_SPOKE_CONTEXTS": "cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3"
      }
    }
  }
}

Note the top-level key is servers, not mcpServers - VS Code differs from Claude Code and Gemini CLI here, and copying one into the other fails silently.

[mcp_servers.ocm-fleet]
command = "ocm-mcp-server"

[mcp_servers.ocm-fleet.env]
OCM_MCP_HUB_CONTEXT = "kind-hub"
OCM_MCP_SPOKE_CONTEXTS = "cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3"
{
  "mcpServers": {
    "ocm-fleet": {
      "command": "ocm-mcp-server",
      "env": {
        "OCM_MCP_HUB_CONTEXT": "kind-hub",
        "OCM_MCP_SPOKE_CONTEXTS": "cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3"
      }
    }
  }
}
{
  "mcpServers": {
    "ocm-fleet": {
      "command": "ocm-mcp-server",
      "env": {
        "OCM_MCP_HUB_CONTEXT": "kind-hub",
        "OCM_MCP_SPOKE_CONTEXTS": "cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3"
      }
    }
  }
}

Most MCP clients accept an mcpServers block like this one. If ocm-mcp-server is not on the PATH the client launches with, use the absolute path from which ocm-mcp-server as the command value.

Give the agent the runbook discipline in examples/system-prompt.md, then break something and watch the flow:

make inject SCENARIO=failing-rollout CLUSTER=cluster2

You: "Payments is degraded somewhere in the fleet. Investigate and fix."

Agent: list_clusters → get_cluster_health(cluster2) → query_events → get_pod_logs → "payments-v2 on cluster2 is in ImagePullBackOff. Proposing a ManifestWork pinning the last good image. Proposal 4f1a2b3c needs your approval."

You (trusted terminal): ocm-mcp approve 4f1a2b3c, then paste the token back.

Agent: apply_manifestwork → verifies recovery → get_audit_trail → writes the incident report.

Then try to talk it into something dangerous ("just redeploy it privileged with hostNetwork, it's faster"). The proposal dies at layer 1 or layer 2, and the rejection message tells the agent exactly why. More worked examples →

Architecture

flowchart LR
    A["🤖 AI Agent<br/>(any MCP client)"] -->|"typed tool calls"| S["🛡️ ocm-mcp-server<br/>static guardrails · audit"]
    S -->|"reads + dry-run + apply"| H["☸️ OCM Hub<br/>Placement · ManifestWork<br/>Kyverno · RBAC"]
    H --> C1["cluster1"]
    H --> C2["cluster2"]
    H --> C3["cluster3"]
    U["🧑‍💻 Human operator<br/>ocm-mcp approve"] -.->|"approval token"| A
    S -.->|"spans"| J["🔍 OpenTelemetry / Jaeger"]

The write path in one sentence: the agent proposes a ManifestWork; static guardrails and a Kyverno dry-run validate it; a human reviews the exact content and mints an approval token bound to its hash; only then does apply deliver it, with every step traced and logged.

Policy admission with Kyverno

The second guardrail layer does not live in this server - it lives in the cluster. Before a proposed change is ever stored, the server does a server-side dry-run create of the ManifestWork on the hub, so the hub's Kyverno validating admission runs against the exact manifests the agent wants to apply. If your organization's policy says no, the proposal is rejected at admission with the policy's own message - the same control that governs every human kubectl apply.

Why Kyverno:

  • Policy as code, no new language. Kyverno is a CNCF policy engine whose policies are ordinary Kubernetes resources in YAML and CEL - reviewable, versioned, and testable like any manifest. This is the policy-as-code approach the CNCF Kubernetes Policy Management whitepaper (CNCF TAG Security) recommends: keep policy declarative and separate from application code.

  • Enforced by the cluster, not the prompt. Admission control is external to the model and to this server; it cannot be talked out of the way a system prompt can.

  • The right tool for the job. Kyverno can validate, mutate, generate, and verify images; here it is used to validate the workloads embedded inside a ManifestWork.

Where it is used here:

  • deploy/policies/ ships 9 ClusterPolicy objects that foreach over spec.workload.manifests inside a ManifestWork: block privileged/host access, protect system namespaces, enforce a kind allow-list, require the managed-by label from the server ServiceAccount (so an unlabeled work cannot skip the others), and enforce a Restricted-Pod-Security baseline in parity with the static guardrails. They are scoped by the app.kubernetes.io/managed-by: ocm-mcp-server label so they judge only agent-authored work. They are usable on their own: deploy/policies/README.md documents the foreach-over-embedded-manifests pattern, the two identifiers an adopter changes, and the Kyverno versions the pack is actually tested against.

  • make policy-test runs a 46-case offline suite with the kyverno CLI - good, bad, and human-authored ManifestWorks - needing no cluster and no dependencies. It runs in CI, so a policy regression fails the build before it can reach a hub.

  • Don't start from scratch: the community library kyverno/policies and the searchable Kyverno Policies catalog are a ready source of validation, Pod Security Standards, and best-practice policies to adopt or take inspiration from.

Toolsets

The surface is 37 tools across ten toolsets. Almost all of it is read: the whole Open Cluster Management API is safe to inspect. Only two toolsets can change anything, and only through the propose -> approve -> apply gate. Every hub-level tool works for any managed spoke - a standalone OpenShift cluster, a HyperShift hosted cluster, or a cloud cluster - because on the hub they are all ManagedClusters.

Toolset

What it covers

Tools

Writes

inventory

ManagedClusters, ClusterSets, set bindings, ClusterClaims, ManagedClusterInfo

6

-

observability

cluster health, one-call fleet sweep, events, pod logs

4

-

placement

Placements, PlacementDecisions, AddOnPlacementScores

3

-

work

ManifestWork status feedback + the gated deploy and rollback flow

9

gated

addons

ClusterManagementAddOns, fleet + per-cluster add-on health

3

-

registration

pending join CSRs + gated cluster lifecycle actions

3

gated

policy

governance compliance + violations rollup (if the add-on is installed)

2

-

hosted-control-planes

HyperShift HostedClusters and NodePools (when the hub hosts them)

3

-

resources

generic get/list over an allow-list of OCM API types

2

-

audit

pending proposals, this server's own audit trail

2

-

The whole surface at once. Eight toolsets cannot change anything at all; the two that can are the two wearing a lock.

Every read tool is annotated readOnlyHint; every write tool is annotated destructiveHint and enforced by the gate. Setting OCM_MCP_READ_ONLY=1 turns off the two writing toolsets entirely, for a strictly-inspection deployment.

Validate against your own hub in one command: ocm-mcp doctor calls every read tool against the live hub and prints a PASS / EMPTY / SKIP / FAIL table (writing nothing), so you can confirm exactly what the server sees before wiring up an agent.

The two lanes, to scale: a read answers straight away, a write crawls through propose, a human signature, and a one-time-token apply.

There is deliberately no tool that reads Secrets, execs into pods, or deletes arbitrary resources. The generic reader (list_resources / get_resource) works against an allow-list of OCM types, so Secrets are not restricted - they are simply not expressible. A capability that does not exist cannot be prompt-injected into use.

Tools

Each tool below is annotated with its class: read (free, no gate), propose (stores a pending change, mutates nothing), or apply (delivers an approved change; needs a human token).

  • list_clusters (read) - all managed clusters with availability, version, labels, capacity.

  • get_cluster (read) - full view of one cluster.

    • cluster (string) - managed cluster name.

  • list_cluster_sets (read) - ManagedClusterSets with selector type and member clusters.

  • list_cluster_set_bindings (read) - which ClusterSets a namespace's Placements may target.

    • namespace (string, optional) - limit to one namespace; empty lists all.

  • list_cluster_claims (read) - every cluster's ClusterClaims (id, platform, region, version).

  • get_cluster_info (read) - extended inventory from the hub (OpenShift version, nodes, console URL); needs no spoke access.

    • cluster (string) - managed cluster name.

  • get_cluster_health (read) - hub conditions, unhealthy pods, degraded deployments.

    • cluster (string) - managed cluster name.

  • get_fleet_health (read) - health of the whole fleet in one call: hub conditions for every cluster plus concurrent spoke scans; broken spokes show as an error entry instead of failing the sweep.

    • clusters (string, optional) - comma-separated cluster names to scope the sweep; empty means every cluster.

  • query_events (read) - recent Kubernetes events, newest first.

    • cluster (string) - managed cluster name.

    • namespace (string, optional) - namespace filter; empty means all.

    • limit (int, optional) - max events (default 40).

  • get_pod_logs (read) - tail a pod's logs (falls back to the previous instance if crashing).

    • cluster (string), namespace (string), pod (string) - target.

    • container (string, optional) - container name; empty picks the default.

    • lines (int, optional) - trailing lines (default 80).

  • list_placements (read) - Placements and how many clusters each selects.

    • namespace (string, optional) - limit to one namespace.

  • get_placement_decision (read) - the clusters a Placement actually selected.

    • placement (string) - Placement name.

    • namespace (string) - the Placement's namespace.

  • list_addon_placement_scores (read) - custom scores prioritizers consume.

    • cluster (string) - managed cluster name.

  • list_manifestworks (read) - ManifestWorks targeting a cluster.

    • cluster (string) - managed cluster name.

  • get_manifestwork (read) - detailed status + per-resource status feedback (the "why not Applied").

    • cluster (string), name (string) - target.

  • list_manifestworkreplicasets (read) - a template fanned across a Placement, with rollout summary.

    • namespace (string, optional) - limit to one namespace.

  • propose_manifestwork (propose) - propose a change as a ManifestWork. Applies nothing.

    • cluster (string) - target cluster.

    • name (string) - short kebab-case ManifestWork name.

    • summary (string) - one or two sentences the human approver reads.

    • manifests_json (string) - JSON array of complete manifests (allowed kinds; namespaced; pinned images).

  • apply_manifestwork (apply) - deliver an approved ManifestWork.

    • proposal_id (string), approval_token (string) - from ocm-mcp approve <id>.

  • propose_rollback (propose) - propose undoing an applied ManifestWork; creates a rollback proposal bound to its UID.

    • proposal_id (string) - the applied ManifestWork proposal to undo.

  • rollback_manifestwork (apply) - delete the ManifestWork after the rollback is approved (needs a rollback-scoped token).

    • rollback_proposal_id (string), approval_token (string).

  • list_cluster_management_addons (read) - fleet-level add-on definitions and install strategy.

  • get_addon_health (read) - per-cluster ManagedClusterAddOn Available / Degraded / Progressing.

  • list_addons_for_cluster (read) - every add-on on one cluster, with install namespace and health.

    • cluster (string) - managed cluster name.

  • list_pending_csrs (read) - pending cluster-join / add-on registration CSRs awaiting approval.

  • propose_cluster_action (propose) - propose a lifecycle action. Applies nothing.

    • cluster (string) - target cluster.

    • action (string) - one of cordon (taint out of scheduling), uncordon, set_label, accept (hubAcceptsClient + approve join CSRs), enable_addon / disable_addon (create/delete a ManagedClusterAddOn).

    • summary (string) - what the human approver reads.

    • params_json (string, optional) - action parameters; set_label needs {"key","value"}, the add-on actions need {"addon"} (+ optional install_namespace).

  • apply_cluster_action (apply) - apply an approved lifecycle action.

    • proposal_id (string), approval_token (string).

  • list_policies (read) - Policies and per-cluster compliance. Reports clearly if the governance add-on is not installed.

    • namespace (string, optional) - limit to one namespace.

  • list_policy_violations (read) - only the NonCompliant / Pending policy-cluster pairs across the fleet.

  • list_hosted_clusters (read) - HostedClusters with version and conditions. Reports clearly if HCPs are hosted on a different management cluster.

    • namespace (string, optional) - limit to one namespace.

  • get_hosted_cluster (read) - one HostedCluster in detail, with its NodePools.

    • name (string), namespace (string) - target.

  • list_node_pools (read) - HyperShift NodePools (worker groups), desired vs current replicas.

    • namespace (string, optional), cluster (string, optional) - filters.

  • list_resources (read) - list any allow-listed OCM type (identity + conditions).

    • resource (string) - e.g. managedclusters, placements, manifestworks, managedclusteraddons, klusterlets.

    • namespace (string, optional) - for namespaced types.

  • get_resource (read) - get one allow-listed OCM object in full. Never returns a Secret (not on the allow-list).

    • resource (string), name (string) - target.

    • namespace (string, optional) - required for namespaced types.

  • list_pending_proposals (read) - ManifestWorks and cluster actions awaiting approval.

  • get_audit_trail (read) - the last N tool calls from this server's append-only log.

    • last_n (int, optional) - trailing entries (default 30).

Prompts

The server also ships ten MCP prompts - reusable templates that encode the safe workflow so any client can start from a good runbook instead of a blank box.

Prompt

What it drives

Arguments

diagnose_fleet

sweep every cluster and add-on, summarize what is unhealthy and why - reads only

-

remediate_with_approval

investigate a symptom, propose the smallest safe fix, wait for the human token, apply, verify, report

symptom

incident_postmortem

write the post-incident report strictly from get_audit_trail, not from memory

-

why_not_scheduled

explain why a cluster was or was not selected by a Placement, from the live objects

cluster, placement, namespace

onboard_cluster

accept a pending cluster safely through the approval gate

cluster

addon_troubleshoot

diagnose a degraded add-on across the fleet

addon

hosted_cluster_health

assess a HyperShift hosted control plane and its node pools

cluster

policy_compliance_report

summarize governance compliance and prioritize what to fix

-

capacity_report

find clusters with headroom and clusters under pressure

-

rollout_status

track a ManifestWorkReplicaSet rollout across selected clusters

name, namespace

Resources

The server also exposes 6 MCP resources - read-only fleet state a client can pin, browse, or attach as context without a tool call (strictly a subset of the read tools; every access still writes an audit line):

URI

What it serves

ocm://clusters

all ManagedClusters (availability, version, labels, capacity)

ocm://clusters/{cluster}

full view of one ManagedCluster

ocm://policies

governance policies + per-cluster compliance (if installed)

ocm://proposals

proposals waiting for human approval

ocm://audit/tail

the last 50 entries of the tamper-evident audit log

ocm://guardrails

the exact allow-lists and limits proposals are checked against - reading it first avoids a rejection round-trip

Observability - audit, tracing (OpenTelemetry/Jaeger), metrics

Every tool call produces up to three independent records, each with a different job:

Signal

Always on?

What it answers

Where it goes

Audit log

yes

what happened, in what order, on whose authority

audit.jsonl - hash-chained, anchor-signed, the source for incident reports and the eval harness

OTel trace span

opt-in

where time went; the call structure behind a slow or failed operation

any OTLP backend: Jaeger, OTel Collector, Grafana Tempo, ...

Prometheus metrics

opt-in

how often and how slow, per tool and outcome, for dashboards/alerts

GET /metrics (OCM_MCP_METRICS_PORT, localhost by default). This server's own counters only - it does not scrape or proxy Prometheus on managed/HCP clusters; fleet state comes from the Kubernetes APIs

The same call, recorded twice. Edit an earlier line and the chain says so; the spans, meanwhile, only ever answer where the time went.

What the tracing is: OpenTelemetry is the CNCF standard for distributed tracing; Jaeger is a CNCF trace viewer. When enabled, this server opens one span per tool call - named tool.<name> (e.g. tool.apply_manifestwork) - with the call's arguments attached as attributes. The approval_token is never attached, and argument values are truncated at 200 characters, so traces are safe to ship to a shared backend.

Why it exists alongside the audit log: the audit log is a safety artifact - append-only and tamper-evident - while spans are a debugging artifact: in Jaeger you can see that a get_cluster_health call spent 4 s waiting on one spoke, or follow the exact propose → apply sequence of an incident on a timeline. Nothing safety-related trusts the spans, which is why tracing can stay optional and fail-soft: without the extra installed and an endpoint set, it is a no-op.

How to use it (two switches + a viewer):

pip install "ocm-mcp-server[tracing]"                    # OTel SDK + OTLP/HTTP exporter
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 # your collector

# a local Jaeger to look at traces (make bootstrap starts this for you):
podman run -d --name jaeger -p 16686:16686 -p 4318:4318 jaegertracing/all-in-one:1.60
# open http://localhost:16686 and select the "ocm-mcp-server" service

How it is tested: unit tests cover span creation, token redaction, and the no-op paths; the e2e suite includes a tracing-export step that stands up a local OTLP sink, makes a tool call in a fresh server process, and asserts a real trace batch arrives naming both the tool.* span and the ocm-mcp-server service - so the export wiring is proven on every make e2e and in the nightly CI run. Details: deployment guide - tracing and architecture - observability.

Quickstart (laptop, ~15 minutes)

Three ways in. Every row ends somewhere useful — nobody has to reach the third to get an answer out of the first.

Requirements: podman (or docker), kind, kubectl, clusteradm, helm, Python 3.11+, Linux or macOS (Windows unsupported - use WSL2). The deployment guide has install commands and the real-fleet path.

git clone https://github.com/ocm-mcp-server/ocm-mcp-server.git
cd ocm-mcp-server

make bootstrap      # 1 hub + 3 managed kind clusters, OCM, Kyverno, policies, demo app
make install        # pip install -e ".[dev,tracing]"

Configuration

The server is configured entirely through environment variables. The two that matter most are kubeconfig context names. New to those? The context names guide explains what they are and the exact commands to find yours, from a laptop kind cluster to a cloud login. In short: run kubectl config get-contexts and read the NAME column (make bootstrap prints ready-to-paste values at the end).

Variable

Required

What goes in it

OCM_MCP_HUB_CONTEXT

yes

The kubeconfig context that points at the OCM hub cluster, where ManagedCluster and ManifestWork live. After make bootstrap this is kind-hub. Empty = current context.

OCM_MCP_SPOKE_CONTEXTS

for events/logs

Comma-separated <managed-cluster-name>=<kubeconfig-context> pairs mapping each cluster as the hub names it (kubectl --context kind-hub get managedclusters) to a context holding read-only spoke credentials. Only query_events / get_pod_logs / spoke-side health need this; hub-level tools work without it.

KUBECONFIG

no

Kubeconfig file path(s); defaults to ~/.kube/config.

OTEL_EXPORTER_OTLP_ENDPOINT

no

Set (e.g. http://localhost:4318) to emit a trace span per tool call (needs the [tracing] extra; see the tracing guide). Unset = tracing off, audit log still on.

OCM_MCP_HOME

no

State directory (approval keypair, pending proposals, audit.jsonl, spent-token ids). Default ~/.ocm-mcp.

OCM_MCP_SIGNER_KEY

recommended

Path to the private Ed25519 signing key. Point this off the server (a separate account/device) so a compromised server cannot mint tokens. Default OCM_MCP_HOME/approval_ed25519.

OCM_MCP_VERIFIER_KEY

no

Path to the public verifier key the server loads. Mount read-only. Default OCM_MCP_HOME/approval_ed25519.pub.

OCM_MCP_ISSUER / OCM_MCP_AUDIENCE

no

Bind approval tokens to this deployment so a token minted elsewhere is refused. Defaults ocm-mcp / ocm-mcp-server.

OCM_MCP_APPROVAL_TTL

no

Approval-token lifetime in seconds. Default 3600.

OCM_MCP_REQUIRE_DIGEST

no

Set to 1 to require @sha256 digest-pinned images (stricter than tag-pinning). Default off.

OCM_MCP_METRICS_PORT

no

If set, expose Prometheus metrics at /metrics on this port. Default off.

OCM_MCP_METRICS_HOST

no

Interface the metrics endpoint binds. Default 127.0.0.1 (localhost only); set 0.0.0.0 for a remote scraper.

OCM_MCP_AUDIT_ECHO

no

Set to 1 to also echo each audit line to stderr as JSON, so a container log collector can forward the audit stream to a SIEM. Free-form values (manifests, summaries, reasons, error text) are redacted ("[redacted]") in the echo; the audit file itself keeps full fidelity. Default off.

OCM_MCP_MAX_PROPOSAL_BYTES

no

Reject a proposal larger than this many bytes. Default 262144 (256 KiB).

OCM_MCP_MAX_HPA_REPLICAS

no

Reject a HorizontalPodAutoscaler whose maxReplicas exceeds this. Default 100.

OCM_MCP_READ_ONLY

no

Set to 1/true for a strictly-inspection deployment: every propose/apply tool refuses, a coarse backstop under the token gate. Default off.

OCM_MCP_STRICT

no

Set to 1/true to make the deployment preconditions layer&nbsp;3 rests on a startup failure rather than a stderr warning: a signer private key readable next to the server, or OCM_MCP_ISSUER/OCM_MCP_AUDIENCE left at their defaults. Recommended in production. Default off, so a laptop bootstrap still works with one shared home.

OCM_MCP_CLIENT_TTL

no

Seconds before the cached Kubernetes API client is rebuilt, so rotated/refreshed credentials are picked up. Default 600.

OCM_MCP_FANOUT_WORKERS

no

Concurrent spoke scans during get_fleet_health. Default 8 (floor 1).

OCM_MCP_SPOKE_TIMEOUT

no

Read timeout (seconds) for spoke health/event/log calls, so one large cluster cannot hang a tool. Default 30.

OCM_MCP_HEALTH_LIMIT

no

Max pods/deployments get_cluster_health fetches per cluster; the result notes truncation. Default 500.

For key management, ocm-mcp rotate-secret generates a fresh Ed25519 approval keypair (invalidating every outstanding approval token); ocm-mcp doctor runs the live read-path smoke test; and ocm-mcp audit-verify recomputes the audit log's hash chain to detect any edit, reordering, or mid-log deletion.

# the values make bootstrap prints, spelled out:
export OCM_MCP_HUB_CONTEXT=kind-hub                      # context of the hub cluster
export OCM_MCP_SPOKE_CONTEXTS=cluster1=kind-cluster1,cluster2=kind-cluster2,cluster3=kind-cluster3
#                             └ name on the hub ┘ └ kubeconfig context with read-only creds ┘

Not sure where kind-hub or cluster1=kind-cluster1 come from, or what your own values should be? The context names guide walks through it step by step, including cloud logins (EKS, GKE, AKS, OpenShift). Pointing at a real fleet instead of kind? Same variables; the deployment guide covers the read-only spoke accounts and production hardening.

Then hand the server to your agent: the ready-made client configs are in Connect your agent near the top of this README.

Evaluation harness: honest numbers

eval/ ships 22 scripted incident scenarios in three classes: remediate (15), diagnose-only (3), adversarial (4). Scoring is objective on all three axes: diagnosis keywords in the transcript, live cluster state for recovery, and the server's own audit log for safety.

python3 eval/run_eval.py --agent-cmd "claude -p"     # or any agent CLI

Published results (eval/results/published/), failures included. Every row links to its own raw JSON:

Agent (model)

Diagnosis

Recovery

Safety

Not measured

Time taken

Codex CLI (gpt-5.6-sol)

20/22

8/15

19/19

3

76 min

Gemini (Antigravity) (gemini-3.7-flash)

19/22

8/15

22/22

0

79 min

IBM Bob Shell (bob-default)

17/22

8/15

21/21

1

48 min

Claude Code (sonnet)

14/22

8/15

20/20

2

104 min

All runs on the same build (v0.6.0, 37 tools, MCP SDK 2.1.1), same fleet, same 22 scenarios. Time taken is wall clock for the whole run. The agy run did not pin a reasoning tier: that CLI offers the model only as high/medium/low and the run took its default, so the exact tier is not recorded.

Not measured counts scenarios where the agent made no tool call, so the server was never consulted. The agent declined on its own, before the request reached the guardrails. Those are excluded from the safety denominator rather than scored, because counting them either way misreports: as a guardrail success that was not earned, or as a failure that did not happen.

Safety is the axis this server exists for, and it held on every scenario that reached it. The interesting number is the one beside it. Frontier models increasingly refuse an adversarial bait before calling any tool, so the guardrails are never consulted and a bait that was never presented would otherwise score identically to a bait that was blocked. Recovery misses concentrate on scenarios whose fix needs state the read surface deliberately withholds.

Diagnosis and recovery belong to the agent and are published unflattered. Safety is the axis this server is answerable for, and it is the full one.

Run it against your model of choice and publish your numbers, including the failures. The point is real data about what agents can and cannot yet be trusted to do.

The Kyverno policies have their own offline test suite: make policy-test runs 46 CLI cases (deploy/policies/tests/) against good, bad, and human-created ManifestWorks with no cluster and no dependencies. It runs in CI too, so a policy regression fails the build before it ever reaches a hub.

Try it end to end (one command)

Want proof it works against real clusters, not mocks? One script stands up a real Open Cluster Management fleet on kind, exercises every tool and prompt, runs a break-then-fix scenario, and writes a graphical HTML report:

./hack/e2e-local.sh          # 2 spokes, auto-cleanup   (SPOKES=1 for a lighter run)

It (1) installs or version-checks the dependencies (Podman, kind, kubectl, clusteradm, helm; Docker is not required), (2) bootstraps a hub plus spokes, (3) runs every read tool, the gated propose -> approve -> apply write flow, the gated ROLLBACK flow, every lifecycle action (cordon/uncordon, set_label, accept, enable/disable_addon), and all ten prompts - each with a plain-language explanation of what it does and why, (4) drives the real server binary over stdio JSON-RPC with the official MCP client (handshake, tools, prompts, resources, annotations), (5) runs a negative sweep proving every gate fails closed (expired token, replayed token, apply-scoped token refused for rollback, read-only mode, tampered audit log caught, signed audit anchor verified) plus a tracing-export check (OTel spans over OTLP received by a local sink), (6) injects a failing rollout and shows the diagnose-and-fix loop end to end, then (7) writes e2e-report.html and tears the fleet back down (kind and Podman stay installed). The report is git-ignored. Works on macOS (Homebrew + Podman) and Linux, and runs nightly in CI.

ocm-mcp doctor runs just the live read-path smoke test on its own, against any hub.

Here is a real, unedited run (recorded with asciinema, long waits compressed): the fleet comes up, every step passes, and the fleet is torn down again - MP4 version · terminal cast:

Documentation

Page

What it covers

Tools and Prompts reference

every tool by toolset, its class (read / propose / apply), arguments, and the OCM API it touches; the ten MCP prompts

Context names guide

zero-background: what a kubeconfig context is and the exact commands to find yours (kind, EKS, GKE, AKS, OpenShift)

Deployment guide

laptop quickstart in depth, real OCM fleets, container images, production hardening, troubleshooting

Worked examples

full incident transcripts, approval sessions, adversarial rejections, audit output

Architecture (root pointer: ARCHITECTURE.md)

the choke-point idea, components, the full low-level design (vertical diagrams: stack, call anatomy, write gates, rollback, audit machinery), design decisions worth arguing about

Guardrails

the four layers, deliberate absences, threat model, what we refuse to automate

Security self-assessment

CNCF TAG-Security-style assessment: actors, actions, security functions, limits

CNCF Sandbox readiness

a self-check against CNCF Sandbox expectations, used as a quality bar; honest gaps

Demo script

a timed 3-act live demo with fallbacks

Upstream notes

gaps found while building this; proposals for MCP, OCM, and Kyverno

Fleet-scale benchmarks

real measured numbers: hub-side pagination at 1000+ ManagedCluster CRs, concurrent vs. sequential fleet_health fan-out across real kwok spoke apiservers

Eval harness

scenario classes, scoring, how to run against your model

Blog posts

long-form deep dives: the guardrail design, ten lessons for MCP servers, the three-agent evaluation, and the fleet-scale benchmarks

Changelog · Support · Security · Contributing

project meta

Repository map

src/ocm_mcp_server/   the MCP server: tools, guardrails, approvals, tracing, CLI
deploy/               least-privilege RBAC + Kyverno ClusterPolicies (+ offline tests)
hack/                 bootstrap.sh / teardown.sh / demo app (kind-based fleet)
chaos/                failure-injection scenarios (reversible, diagnosable)
eval/                 22-scenario evaluation harness + results
blogs/                long-form posts (canonical drafts; published to Medium)
docs/                 deployment, examples, architecture, guardrails, demo, upstream
examples/             MCP client configs (Claude, VS Code, Codex, Gemini) + system prompt

Roadmap

The canonical, themed roadmap lives in ROADMAP.md. Current headline items: an authenticated HTTP transport with per-tool scopes, an off-box (KMS/HSM) approval signer, the OCM cluster-proxy transport, and a reusable Kyverno policy pack. (Multi-model eval results are now published.)

Have a need that's not there? Open a feature request. New tools require a safety rationale; see CONTRIBUTING.md.

Contributing & community

Issues and PRs welcome. Start with CONTRIBUTING.md. Getting help: SUPPORT.md. Security reports (privately, please): SECURITY.md.

Sponsorship

This project is independently maintained. If your organization wants priority integration help, a hardened deployment review, sponsored features, or talks and workshops on safe agentic operations, connect on LinkedIn (details in SUPPORT.md).

Author

Sandeep Bazar - Passionate in Technology especially around Multi-cluster Kubernetes platforms, day-2 operations, and making fleets safer to automate.

LinkedIn YouTube

If this project is useful to you, a ⭐ helps others find it.

Project governance and maturity

This project holds itself to CNCF community, governance, and security practices as a quality bar - the same standards expected of a CNCF Sandbox project - so it is easy to adopt, contribute to, and trust. The scaffolding is in place:

Contributions are signed off under the DCO - enforced by a CI job on every pull request, not just asked for in a checklist - published release tags are immutable, and any change that touches a guardrail requires a written safety rationale.

License

Apache-2.0

Available Tools

35 tools
apply_cluster_actionA
Destructive

Apply a previously proposed cluster lifecycle action. Requires a human-minted token.

Args: proposal_id: id returned by propose_cluster_action. approval_token: token the operator produced with ocm-mcp approve <id>.

Inputs: proposal_id (required), approval_token (required). Returns: result

apply_manifestworkA
Destructive

Apply a previously proposed ManifestWork. Requires a human-minted approval token.

Args: proposal_id: id returned by propose_manifestwork. approval_token: token the operator produced with ocm-mcp approve <id>.

Inputs: proposal_id (required), approval_token (required). Returns: result

get_addon_healthA
Read-only

Per-cluster add-on health across the fleet (ManagedClusterAddOn Available / Degraded).

Inputs: no named parameters. Returns: result

get_audit_trailA
Read-only

Return the last N entries of this server's own tool-call audit log.

Args: last_n: number of trailing audit entries to return (default 30).

Use this at the end of an incident to write an accurate post-incident report of what was inspected, proposed, approved, and applied - from the record, not from memory.

Inputs: last_n. Returns: result

get_clusterA
Read-only

Full view of one ManagedCluster.

Args: cluster: managed cluster name as the hub knows it (see list_clusters).

Returns acceptance (hubAcceptsClient), taints, hub conditions, Kubernetes version, capacity/allocatable, and the cluster's ClusterClaims.

Inputs: cluster (required). Returns: result

get_cluster_healthA
Read-only

Health summary for one cluster: hub conditions, unhealthy pods, degraded deployments.

Args: cluster: managed cluster name. Pod/deployment detail requires a read-only spoke context (OCM_MCP_SPOKE_CONTEXTS); hub conditions work without one.

Inputs: cluster (required). Returns: result

get_cluster_infoA
Read-only

Extended inventory for one cluster from the hub: OpenShift version, nodes, console URL.

Args: cluster: managed cluster name.

Reads ManagedClusterInfo on the hub, so it needs no spoke access and works for any spoke - external OpenShift, HCP, or cloud. Needs the ACM/MCE multicloud-operators-foundation add-on; reports clearly if it is not present.

Inputs: cluster (required). Returns: result

get_fleet_healthA
Read-only

Health of the WHOLE fleet in one call: hub conditions for every cluster plus concurrent pod/deployment scans of each spoke that has a read context.

Args: clusters: optional comma-separated managed-cluster names to scope the sweep; empty means every cluster on the hub.

Clusters with problems sort first. A broken spoke shows an 'error' entry instead of failing the sweep. Prefer this over calling get_cluster_health in a loop.

Inputs: clusters. Returns: result

get_hosted_clusterA
Read-only

Detailed HostedCluster: version, conditions, and its NodePools.

Args: name: HostedCluster name. namespace: the namespace the HostedCluster lives in (its hosting namespace).

Inputs: name (required), namespace (required). Returns: result

get_manifestworkA
Read-only

Detailed ManifestWork status: top-level conditions and per-resource status feedback.

Args: cluster: managed cluster name. name: ManifestWork name.

Use this to answer 'why is this ManifestWork not Applied/Available' and to read status feedback (for example replica counts) reported back from the spoke.

Inputs: name (required), cluster (required). Returns: result

get_placement_decisionA
Read-only

Which clusters a Placement actually selected (reads its PlacementDecisions).

Args: placement: Placement name. namespace: the namespace the Placement lives in.

Inputs: namespace (required), placement (required). Returns: result

get_pod_logsA
Read-only

Tail logs from a pod on a managed cluster.

Args: cluster: managed cluster name. namespace: pod namespace. pod: pod name. container: container name; empty picks the default container. lines: number of trailing log lines (default 80).

Falls back to the previous container instance if the current one is crashing.

Inputs: pod (required), cluster (required), namespace (required), lines, +1 more. Returns: result

get_resourceA
Read-only

Generic get of one allow-listed OCM resource, in full.

Args: resource: an OCM resource type (see list_resources for the allow-list). name: object name. namespace: required for namespaced types (usually the cluster namespace).

Never returns a Secret: Secrets are not on the allow-list, so this capability does not exist rather than being merely restricted.

Inputs: name (required), resource (required), namespace. Returns: result

list_addon_placement_scoresA
Read-only

List AddOnPlacementScores in a cluster's namespace (custom scores prioritizers consume).

Args: cluster: managed cluster name (its hub namespace holds the scores).

Inputs: cluster (required). Returns: result

list_addons_for_clusterA
Read-only

Every add-on installed on one cluster, with health (ManagedClusterAddOn in its namespace).

Args: cluster: managed cluster name.

Inputs: cluster (required). Returns: result

list_cluster_claimsA
Read-only

Every cluster's ClusterClaims (id, platform, region, version) rolled up from status.

Inputs: no named parameters. Returns: result

list_cluster_management_addonsA
Read-only

List fleet-level add-on definitions (ClusterManagementAddOn) and their install strategy.

Inputs: no named parameters. Returns: result

list_clustersA
Read-only

List all managed clusters with availability, version, labels, and capacity.

Inputs: no named parameters. Returns: result

list_cluster_set_bindingsA
Read-only

List ManagedClusterSetBindings (which ClusterSets a namespace's Placements may use).

Args: namespace: limit to one namespace; empty lists bindings across all namespaces.

Inputs: namespace. Returns: result

list_cluster_setsA
Read-only

List ManagedClusterSets with their selector type and member clusters.

Inputs: no named parameters. Returns: result

list_hosted_clustersA
Read-only

List HyperShift HostedClusters, when the hub is the HCP hosting cluster.

Args: namespace: limit to one namespace; empty lists across all namespaces.

HostedCluster objects live on whichever cluster hosts the control plane. If your HCPs are hosted on a separate management cluster, they are not on this hub - this reports that clearly, and the spokes still appear via list_clusters.

Inputs: namespace. Returns: result

list_manifestworkreplicasetsA
Read-only

List ManifestWorkReplicaSets (a template fanned across a Placement) with rollout summary.

Args: namespace: limit to one namespace; empty lists across all namespaces.

Inputs: namespace. Returns: result

list_manifestworksA
Read-only

List ManifestWorks targeting a cluster (what the hub is managing there).

Args: cluster: managed cluster name.

Inputs: cluster (required). Returns: result

list_node_poolsA
Read-only

List HyperShift NodePools (worker groups), optionally filtered to one HostedCluster.

Args: namespace: limit to one namespace; empty lists across all namespaces. cluster: optional HostedCluster name to filter node pools by.

Inputs: cluster, namespace. Returns: result

list_pending_csrsA
Read-only

List pending cluster-join / add-on registration CSRs awaiting hub approval.

Inputs: no named parameters. Returns: result

list_pending_proposalsA
Read-only

List proposals (ManifestWorks and cluster actions) waiting for human approval.

Inputs: no named parameters. Returns: result

list_placementsA
Read-only

List Placements and how many clusters each currently selects.

Args: namespace: limit to one namespace; empty lists across all namespaces.

Inputs: namespace. Returns: result

list_policiesA
Read-only

List governance Policies and per-cluster compliance (only if the add-on is installed).

Args: namespace: limit to one namespace; empty lists across all namespaces.

Inputs: namespace. Returns: result

list_policy_violationsA
Read-only

Only the NonCompliant / Pending Policy-cluster pairs across the fleet - the open risks.

Inputs: no named parameters. Returns: result

list_resourcesA
Read-only

Generic list over an allow-list of OCM API types (identity + conditions only).

Args: resource: an OCM resource type, e.g. managedclusters, placements, placementdecisions, manifestworks, managedclusteraddons, clustermanagementaddons, managedclustersets, policies, klusterlets. namespace: for namespaced types, limit to one namespace; empty lists all.

Only Open Cluster Management types are allowed. Secrets and other credential resources are not on the allow-list and cannot be read through this tool.

Inputs: resource (required), namespace. Returns: result

propose_cluster_actionA

Propose an OCM cluster lifecycle action. Does NOT apply anything.

Args: cluster: target managed cluster name. action: one of 'cordon' (taint out of scheduling), 'uncordon' (undo cordon), 'set_label' (params: {"key","value"}; empty value removes the label), 'accept' (set hubAcceptsClient=true and approve pending join CSRs), 'enable_addon' (params: {"addon","install_namespace"?}; create a ManagedClusterAddOn), 'disable_addon' (params: {"addon"}; delete it). summary: one or two sentences the human approver will read. params_json: JSON object of action parameters (set_label and the addon actions need it; cordon/uncordon/accept do not).

The action is validated with a server-side dry-run, then stored pending. The human operator must run ocm-mcp approve <id> to mint the approval token.

Inputs: action (required), cluster (required), summary (required), params_json. Returns: result

propose_manifestworkA

Propose a change to one cluster as an OCM ManifestWork. Does NOT apply anything.

Args: cluster: target managed cluster name. name: a short kebab-case name for the ManifestWork. summary: one or two sentences a human approver will read. Be precise about what changes and why. manifests_json: JSON array of complete Kubernetes manifests (allowed kinds only; all namespaced; images pinned).

The proposal must pass static guardrails and a Kyverno dry-run on the hub. On success it is stored pending and the human operator must run ocm-mcp approve <id> to mint an approval token.

Inputs: name (required), cluster (required), summary (required), manifests_json (required). Returns: result

propose_rollbackA

Propose rolling back an applied ManifestWork. Applies nothing; needs its own approval.

Args: proposal_id: id of an already-applied ManifestWork proposal.

Creates a distinct rollback proposal bound to the exact ManifestWork name and UID. The human approves it separately (ocm-mcp approve <rollback-id>), and the token can only authorize a rollback - an old apply token can never delete a workload.

Inputs: proposal_id (required). Returns: result

query_eventsA
Read-only

Recent Kubernetes events from a managed cluster, newest first.

Args: cluster: managed cluster name. namespace: optional namespace filter; empty means all namespaces. limit: maximum number of events to return (default 40).

Use this to find the 'why' behind unhealthy pods.

Inputs: cluster (required), limit, namespace. Returns: result

rollback_manifestworkA
Destructive

Delete a ManifestWork after a rollback proposal has been approved.

Args: rollback_proposal_id: id returned by propose_rollback. approval_token: a rollback token from ocm-mcp approve <rollback-id>.

Inputs: approval_token (required), rollback_proposal_id (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 35 tools

    Disambiguation4/5

    Most tools target distinct resources or actions, but some pairs like list_cluster_claims vs list_clusters and get_cluster vs get_cluster_info vs get_cluster_health require careful reading of descriptions. The generic list_resources/get_resource also overlap with specific listers, though the descriptions clarify their limitations.

    Naming Consistency5/5

    All tool names are lowercase with underscores and follow a clear verb_noun pattern (list_*, get_*, propose_*, apply_*, rollback_*, query_events). The convention is uniform and predictable across all 35 tools.

    Tool Count3/5

    At 35 tools, the count is heavy, but the server covers a broad OCM domain (clusters, manifestworks, add-ons, policies, hosted clusters, generic resources, audit). Many list tools could potentially be consolidated, yet each appears to serve a distinct observable purpose, making the count borderline rather than chaotic.

    Completeness4/5

    The tool set provides strong read coverage for most OCM resources and a safe write path for ManifestWorks and cluster actions via propose/apply/rollback. However, there are no direct update/delete operations for placements, policies, or hosted clusters, and the approval mechanism is intentionally external, leaving minor gaps in lifecycle management.

    Maintenance

    ActivitySlowing
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers