Skip to main content
Glama

MCP Control Plane

A hardened gateway that sits between AI agents (LLM clients) and real infrastructure APIs (Kubernetes, Terraform, Jenkins, Prometheus, ticketing). Every tool call is authenticated, rate-limited, policy-checked via OPA, and audited before it reaches an executor. The model is treated as an untrusted caller — the policy engine, not the prompt, is the actual security boundary.

Full target architecture, threat model, and API design live in docs/; this README covers what's implemented today and how to run it.

Status

Actively under development, tracked in docs/roadmap.md. Currently implemented:

  • MCP JSON-RPC gateway (tools/list, tools/call) over HTTP, plus an SSE endpoint that also carries approval-decision pushes

  • API key authentication, resolving each key to an agent identity (id, role, allowed_tools)

  • Redis-backed sliding-window rate limiting, per agent

  • Tool call interception, normalization, and an OPA policy check (allow / deny / require_approval)

  • 7 read-only tool executors calling real downstream APIs (Kubernetes, Terraform Cloud, Jenkins, Prometheus, PagerDuty/Jira), plus the 3 write tools (restart_deployment, scale_deployment, trigger_jenkins_job) — docs/roadmap.md Phases 3-4.

  • 2 higher-risk stretch tools, exec_into_pod and apply_k8s_manifest (docs/roadmap.md Phase 10): same Python kubernetes client, sre1-only, hard-blocked against kube-system, and gated behind approval in every environment (not just prod, unlike the write tools above).

  • Human-in-the-loop approval gate for require_approval policy decisions: pending approvals are persisted to Redis with a 15-minute TTL, a Slack Incoming Webhook notifies with approve/deny buttons, and POST /admin/approvals/{id}/decide (HMAC-verified, replay-protected) resumes the tool call on approval or logs approval_denied on denial — see docs/roadmap.md Phase 4.

  • PostgreSQL-backed, SHA-256 hash-chained audit log: every tool call (allowed, denied, errored, pending/decided approval) gets one append-only row via app/audit.py, schema-migrated with Alembic. GET /admin/audit queries it with filters/pagination and an integrity_check that recomputes the hash chain; GET /admin/audit/export streams the same data as NDJSON. A daily script/workflow exports the prior day's rows to S3 — see docs/roadmap.md Phase 5.

  • OpenTelemetry traces + metrics across the whole request path (authn → policy_eval → executor → audit_write spans, tool_calls_total/policy_denials_total/ approval_requests_total/rate_limit_hits_total/tool_call_duration_ms and friends), an OTel collector + Tempo + Prometheus + Grafana stack in docker-compose.yaml, 6 provisioned Grafana dashboards and 4 alert rules under observability/ — see docs/roadmap.md Phase 6.

  • Terraform for the full AWS deployment target (VPC, EKS + managed node group, ECR, RDS, ElastiCache Redis, ALB/ACM, IRSA IAM roles, Secrets Manager, an Object-Locked S3 bucket for the audit export) plus a Helm chart (helm/mcp-control-plane/) installed by Terraform itself, under terraform/ — see docs/roadmap.md Phase 7, including its "what's actually been checked" note: this was written with no AWS account available, so it's fmt/reference/init-clean and the Helm chart is lint/template-verified, but has never been plan/apply/destroy-run against real AWS.

Not yet built: admin JWT auth in front of /admin/approvals/* and /admin/audit* (both currently have no auth of their own beyond the approval-decide endpoint's Slack HMAC check), automated retention enforcement (the 90-day Postgres deletion and S3-to-Glacier lifecycle rule are documented but not scheduled anywhere), and the EKS DaemonSet variant of the OTel collector (Phase 6 only ships the Docker Compose one; terraform/helm.tf runs the collector as a regular in-cluster Deployment, not a DaemonSet) — see the roadmap for sequencing.

Related MCP server: Pincer-MCP

Architecture

Agent → POST /mcp → [log] → [authn] → [rate limit] → interceptor → OPA → executor → audit log
  • app/main.py — FastAPI app, route handlers, middleware wiring

  • app/middleware/auth.py — API key → agent identity

  • app/middleware/rate_limit.py — Redis sliding-window limiter

  • app/interceptor.py — validates/normalizes tools/call params into an OPA input document

  • app/authz/opa.py — calls the OPA sidecar, maps its response to allow/deny/require_approval

  • app/tools/tools_spec.py — tool executors, 7 read-only + 5 write (all call real downstream APIs)

  • app/tools/k8s_client.py — cached K8s API clients (CoreV1Api/AppsV1Api + a generic DynamicClient for apply_k8s_manifest) + ApiException/timeout -> ExecutorError mapping

  • app/tools/errors.py, app/tools/config.py — shared ExecutorError type and the 10s/30s executor timeout

  • app/approvals.py — Redis-backed pending-approval store (create / decide, 15-minute TTL)

  • app/slack.py — Slack Incoming Webhook notification + X-Slack-Signature HMAC verification

  • app/sse_hub.py — in-process pub/sub pushing an approval decision to the agent's open /mcp/sse connection

  • app/db.py — shared asyncpg (Postgres) connection pool

  • app/audit.py — hash-chained audit log: async writer, filtered query, integrity check, NDJSON export

  • migrations/ — Alembic migrations (raw SQL, no ORM) for audit_log and approvals

  • scripts/export_audit_to_s3.py — daily S3 export of the prior day's audit rows

  • policies/ — Rego policy (authz.rego) and role/tool data (data.json) loaded by OPA

See docs/architecture.md for the full target-state design (EKS, Terraform, Postgres audit log, approval gate, observability stack).

Tech stack

Python 3.13, FastAPI, Redis, OPA/Rego, Docker Compose. See pyproject.toml for exact dependency versions.

Getting started

Prerequisites

  • Python 3.13+ and uv

  • Docker (for Redis/Postgres/OPA via Compose), or your own local instances

Install

uv sync --extra export

(--extra export pulls in boto3 — tests/test_export_script.py imports scripts/export_audit_to_s3.py unconditionally, so a bare uv sync leaves the full uv run pytest run unable to even collect.)

Run

Everything (gateway + Redis + Postgres + OPA + the OTel collector/Tempo/Prometheus/Grafana observability stack) via Docker Compose:

docker compose up

Compose doesn't run the Alembic migration for you - apply it once, as a one-off container on the same network (postgres's port isn't published to the host by default, so this is simpler than reaching it from outside Compose):

docker compose run --rm gateway alembic upgrade head

Grafana is at http://localhost:3000 (anonymous admin access, no login screen - local dev only). The 6 dashboards and 4 alert rules under observability/grafana/ are provisioned automatically. The PagerDuty/Slack contact points (observability/grafana/provisioning/alerting/contact-points.yaml) ship with placeholder values and will provision cleanly either way - replace them with a real integration key/webhook URL before an alert needs to actually page anyone.

Dashboards start empty - they only show data once something sends real traffic through the gateway container (the one docker compose starts; only it has OTEL_EXPORTER_OTLP_ENDPOINT set - uv run pytest and a locally-run uvicorn deliberately don't export telemetry, see app/otel.py). With the stack up and migrated, fire a few requests at it (each maps to a different dashboard/panel) and wait ~60-90s for the metrics export + Prometheus scrape interval:

# Tool Call Volume + Latency - any tool call, success or failure, records these
curl -X POST http://127.0.0.1:8000/mcp -H "Content-Type: application/json" -H "x-api-key: test_key" \
  -d '{"jsonrpc":"2.0","id":"1","method":"tools/call","params":{"name":"get_pod_logs","arguments":{"namespace":"payments","pod_name":"checkout-api-xyz"}}}'

# Denial Rate - has to actually reach OPA and be denied there; a tool the agent isn't
# scoped to at all (app/middleware/auth.py's allowed_tools) gets rejected before OPA/audit
# ever see it and won't show up here - this hits OPA's kube-system hard-block instead
curl -X POST http://127.0.0.1:8000/mcp -H "Content-Type: application/json" -H "x-api-key: test_key" \
  -d '{"jsonrpc":"2.0","id":"2","method":"tools/call","params":{"name":"restart_deployment","arguments":{"namespace":"kube-system","deployment_name":"coredns","reason":"testing denial rate"}}}'

# Approval Queue - require_approval (any write tool in a prod-* namespace)
curl -X POST http://127.0.0.1:8000/mcp -H "Content-Type: application/json" -H "x-api-key: test_key" \
  -d '{"jsonrpc":"2.0","id":"3","method":"tools/call","params":{"name":"restart_deployment","arguments":{"namespace":"prod-payments","deployment_name":"checkout-api","reason":"testing approval queue"}}}'

# Rate Limit - test_key's default limit is 60/min (DEFAULT_RATE_LIMIT_RPM), so fire more than that quickly
for i in $(seq 1 70); do
  curl -s -o /dev/null -X POST http://127.0.0.1:8000/mcp -H "Content-Type: application/json" -H "x-api-key: test_key" \
    -d '{"jsonrpc":"2.0","id":"'$i'","method":"tools/list","params":{}}'
done

Audit Chain Integrity is the one dashboard these won't touch - it only reports data when GET /admin/audit actually finds a broken hash chain, so staying at zero is the correct/healthy state, not something to artificially populate. On Windows, run the block above from PowerShell or Git Bash, not cmd.exe - cmd.exe doesn't treat '...' as a string delimiter the way these commands assume, so the JSON body arrives with literal quote characters in it and the gateway correctly rejects it with -32700 Parse error (same failure mode testing/testing.text 3.8 tests on purpose, just not on purpose here).

Or the gateway alone against local dependencies (Redis, Postgres, OPA each running separately). Apply the audit-log schema once before the first run:

uv run alembic upgrade head
uv run uvicorn app.main:app --host 127.0.0.1 --port 8000 --reload

The app reads REDIS_URL (default redis://localhost:6379), DATABASE_URL (default postgresql://postgres:postgres@localhost:5432/mcp_control_plane), and OPA_URL (default http://localhost:8181/v1/data/authz) from the environment; Compose sets all three to point at the sibling containers. alembic upgrade head reads the same DATABASE_URL.

Tool executor configuration

Each read-only executor talks to a real downstream API and reads its own credentials/URL from the environment (all optional locally — a tool without its config set returns an upstream_error rather than crashing):

Tool(s)

Env vars

get_pod_logs, list_pods, get_deployment_status

none — uses in-cluster config, or KUBECONFIG / ~/.kube/config locally

query_terraform_plan

TFC_TOKEN, TFC_ORG, TFC_URL (default https://app.terraform.io/api/v2)

get_jenkins_job_status

JENKINS_URL, JENKINS_USER, JENKINS_API_TOKEN

read_prometheus_metrics

PROMETHEUS_URL (default http://localhost:9090)

read_ticket (pagerduty)

PAGERDUTY_API_TOKEN, PAGERDUTY_URL (default https://api.pagerduty.com)

read_ticket (jira)

JIRA_URL, JIRA_USER, JIRA_API_TOKEN

all of the above

EXECUTOR_TIMEOUT_SECONDS (default 10, clamped to a 30 max)

Local Kubernetes tools: point KUBECONFIG at a local cluster (e.g. kind create cluster or minikube start, then kubectl config use-context kind-kind / minikube) before calling get_pod_logs, list_pods, or get_deployment_status — the gateway uses whatever context is currently active, same as kubectl.

Approval gate configuration

Env var

Purpose

SLACK_WEBHOOK_URL

Incoming Webhook URL the approval notification is POSTed to. Unset locally: the notification is skipped (logged, not sent) rather than failing the call.

SLACK_SIGNING_SECRET

Shared secret verified against X-Slack-Signature on POST /admin/approvals/{id}/decide. Unset: every callback is rejected with 401 (fails closed, same as OPA being unreachable).

The pending-approval record itself lives in the same Redis as the rate limiter (REDIS_URL), with a 15-minute TTL — no separate store to configure.

Audit log S3 export configuration

scripts/export_audit_to_s3.py (run daily by .github/workflows/audit-export.yml) needs:

Env var

Purpose

S3_BUCKET

Target bucket. No default — the script refuses to run without one rather than silently exporting nowhere.

DATABASE_URL

Same variable app/db.py reads.

AWS credentials

Via boto3's standard chain (env vars, ~/.aws/credentials, or an IAM role in CI).

The GitHub Actions workflow reads these from AUDIT_LOG_S3_BUCKET/AUDIT_LOG_DATABASE_URL/etc. repo secrets and no-ops if they're not configured — there's no deployed Postgres/S3 for this project yet (Phase 7, Infrastructure as Code, hasn't started).

Try it

All /mcp routes require an x-api-key header. The seeded dev key test_key resolves to agent01 (role sre1) with a handful of allowed tools — see API_KEYS in app/middleware/auth.py.

curl -s -X POST http://127.0.0.1:8000/mcp \
  -H "Content-Type: application/json" -H "x-api-key: test_key" \
  -d '{"jsonrpc":"2.0","id":"1","method":"tools/list","params":{}}'

curl -s -X POST http://127.0.0.1:8000/mcp \
  -H "Content-Type: application/json" -H "x-api-key: test_key" \
  -d '{"jsonrpc":"2.0","id":"2","method":"tools/call","params":{"name":"get_pod_logs","arguments":{"namespace":"payments","pod_name":"checkout-api-xyz"}}}'

curl -s http://127.0.0.1:8000/health/live

More request/response examples (error cases, SSE) are in testing/testing.text.

Testing

uv run pytest

Covers the auth middleware, rate limiter, OPA integration, and end-to-end /mcp request handling (see tests/). By default record_tool_call/create_db_pool are stubbed (no real Postgres needed), the same way evaluate_policy is stubbed to allow everything (no real OPA needed) — see tests/conftest.py. Two subsets need Docker and are skipped automatically if it isn't available:

  • tests/test_opa_docker_integration.py — real openpolicyagent/opa image against policies/

  • tests/test_audit_docker_integration.py — real postgres:16-alpine, with the actual alembic upgrade head migration applied, exercising the hash chain (including a tamper/detect test) and the query/export endpoints for real

Rego policy tests run separately, via opa test:

docker run --rm -v "$(pwd)/policies:/policies" openpolicyagent/opa:latest test /policies -v

On Windows, run that from PowerShell, not Git Bash — Git Bash's MSYS layer auto-translates the leading /policies in -v "$(pwd)/policies:/policies" into a bogus Windows path (stat /Program Files/Git/policies: no such file or directory), which has nothing to do with the policies themselves:

docker run --rm -v "${PWD}\policies:/policies" openpolicyagent/opa:latest test /policies -v

opa test runs in CI on every PR (see .github/workflows/ci.yml); the Python test suite above is not currently wired into CI.

Documentation

Doc

Contents

docs/roadmap.md

Phased build plan and current progress

docs/architecture.md

Full target-state system design

docs/api-design.md

MCP + admin API surface

docs/tool-spec.md

Per-tool input/output schemas and policy constraints

docs/threat-model.md

STRIDE threat model and mitigations

docs/rego-policies.md

Rego policy structure, for policy authors

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      A secure gateway that lets AI agents interact with APIs and SSH servers via scoped tokens, with credential injection, human-in-the-loop for dangerous commands, and MCP integration for tool access.
      15
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      A security-hardened MCP gateway that enables AI agents to call LLM APIs (Gemini, OpenAI, Claude, etc.) using ephemeral proxy tokens, eliminating exposure of real API keys.
      43 npm
      6
      Apache 2.0
    • A
      license
      Not graded
      quality
      D
      maintenance
      Provides a secure MCP gateway for AI agents to access APIs without exposing raw credentials, with scoped access, audit logging, and OAuth support.
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Governed MCP gateway that lets AI agents call tools with policy enforcement, prompt-injection screening, a kill-switch, and tamper-evident signed audit logs.
      Apache 2.0