Skip to main content
Glama

gitops-oncall-mcp

License: MIT Python

An MCP server that gives an LLM agent a typed, auditable on-call surface over a GitOps-on-Kubernetes platform: Kubernetes, Argo CD, Argo Rollouts, Prometheus, Loki, Tempo and GitHub.

Requires a Kubernetes cluster. The resource tools read cAdvisor and kube-state-metrics per pod, and the delivery tools read Argo CD and Argo Rollouts custom resources, so there is no meaningful way to run this against plain hosts.

Status: work in progress. Forked from lgtm-oncall-mcp, which targets a hosted Grafana LGTM stack. This one targets a cluster you run yourself, and adds the Kubernetes and Argo surface the other project has no reason to carry.

Why an MCP server and not a shell

The easy version of an on-call agent hands a model kubectl and a terminal. It demos well and is indefensible, because the agent's capability becomes "anything the kubeconfig can do", and the only thing standing between a hallucination and a deleted namespace is the model's judgement.

This inverts it. The agent gets a fixed set of typed tools and nothing else. What it is able to do becomes a code review question rather than a prompting question.

The layering is also the folder structure:

Layer

Enforced by

Where

What the identity can do

the cluster

a ServiceAccount with read verbs only

What the agent can express

this code

tools/senses.py reads, tools/hands.py acts

What a human agreed to

runtime

approval.py, a one-shot proposal id with a TTL

What happened

after the fact

audit.py

Related MCP server: mcp-devops-server

The property worth stating

The agent never writes to the cluster. It writes to Git.

Its ServiceAccount carries no write verbs at all, so mutation is unavailable to it even if every other layer fails. A change it proposes becomes a pull request against the config repository, which Argo CD then syncs and Argo Rollouts then canaries. Every safety mechanism the platform already has applies to the agent for free, and undoing it is git revert, exactly as it would be for a human.

Tools

Reads, no side effects:

Tool

Source

Returns

get_error_rate

Prometheus

5xx as a percentage of all requests, or None when the window held no traffic

get_latency_p95

Prometheus

p95 request duration in seconds, or None when the window held no traffic

get_cpu_usage

Prometheus

cores per pod, against that pod's own limit

get_memory_usage

Prometheus

working-set MiB per pod, against that pod's own limit

get_disk_usage

Prometheus

root filesystem percent per node

get_active_alerts

Alertmanager, else Prometheus

firing alerts, worst severity first

search_logs

Loki

log lines matching a substring in a window, labelled by pod

get_pods

Kubernetes

phase, readiness, restarts, age, and what looks wrong

get_events

Kubernetes

recent warnings, within a recency window

get_rollouts

Argo Rollouts

running image tag, canary step, whether traffic is fully shifted

get_analysis_runs

Argo Rollouts

the verdict that advanced or aborted a canary, and why

get_argocd_apps

Argo CD

sync status, health, target revision, last sync

get_recent_deploys

GitHub

release tags across every application repo, newest first

get_commit_diff

GitHub

the diff for one commit

get_file_commits

GitHub

history for one path

Two of these carry a deliberate warning in their docstring, because the obvious reading is wrong. Events are historical, so an event from hours ago may already be resolved. And an Argo CD Application can read Synced while a rollout is aborted, because Argo CD compares the spec and the refusal lives in the rollout's status.

Writes, split in two so a single confused turn cannot act:

Propose

Confirm

Effect

propose_rollback

confirm_rollback

open a pull request pinning one service to an older tag

propose_pr_change

confirm_pr_change

open a pull request against the config repository

propose_* has no side effect and returns a proposal id with a TTL. confirm_* refuses without a live id. Both ends are written to the audit log.

Planned, not built yet:

Source

Reads

Tempo

traces and span timings

A rollback is a one-line change to the environment's values file, not a pipeline trigger: propose_rollback reads the tag pinned right now and returns both ends of the move, so the human approving it sees v1.0.11 -> v1.0.9 before anything happens. The file is edited line-by-line rather than through a YAML round-trip, which would reformat the whole file and bury a one-line change in an unreadable diff.

Configuration

Every setting is an environment variable. Copy .env.example to .env and fill it in; the file documents each one.

The observability endpoints are addressed directly rather than through a Grafana datasource proxy, because in a cluster they are Services. That removes a hop, a dependency and a credential.

Variable

Required

Purpose

PROMETHEUS_URL

yes

Prometheus HTTP API

LOKI_URL

yes

Loki HTTP API

ALERTMANAGER_URL

no

Alertmanager HTTP API. Unset falls back to Prometheus, which evaluates the alerts but cannot see silences

OBSERVABILITY_TOKEN

no

bearer sent to all three, for setups behind an auth proxy

K8S_NAMESPACE

yes

the one namespace the Kubernetes tools may read

ARGOCD_NAMESPACE

no

where Argo CD runs, default argocd

ENV_LABEL_KEY

no

label that identifies the environment, default env; set to namespace when environments are namespaces

ENV_VALUE_MAP

no

maps the environment names the agent uses onto the label values your metrics carry

GITHUB_TOKEN

yes

fine-grained PAT, scoped to the repositories below and nothing else

GITHUB_OWNER, GITHUB_REPO

yes

the config repository, where pull requests are opened

GITHUB_APP_REPOS

no

application repositories, which is where release tags live

GITHUB_VALUES_PATH

no

values file per environment, {env} substituted; default envs/{env}/values.yaml

MCP_BEARER_TOKEN

when not loopback

shared secret required on every request

The namespace is configuration rather than a tool argument on purpose: the model chooses what to ask about, never what it has access to.

Running

python3 -m venv .venv && .venv/bin/pip install -e .
set -a && . ./.env && set +a
.venv/bin/gitops-oncall-mcp

The server speaks MCP over HTTP on MCP_PORT, default 8765. Point an MCP client at http://127.0.0.1:8765/mcp.

To develop against a real cluster from a laptop, port-forward Prometheus and Loki and set the URLs to localhost.

Development

python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
.venv/bin/pytest -q
.venv/bin/ruff check .

Tests use respx to record HTTP, so the suite needs no cluster and no credentials.

Licence

MIT. See LICENSE.

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    C
    maintenance
    An AI SRE copilot for Kubernetes that lets LLM agents inspect cluster health and, with explicit human confirmation, restart or scale deployments.
    2
    -
  • F
    license
    Not graded
    quality
    D
    maintenance
    A Kubernetes diagnostic agent that provides on-demand root cause analysis and human-in-the-loop remediation via Slack, using LLM reasoning with OPA-bounded security controls.
    1
    -