mcp-kafka-gouverne
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
mcp-kafka-gouverne
An LLM agent plugged into a Kafka bus without breaking the bus. A triage consumer grades night-patrol reports with a model and raises alerts; an MCP server lets people's agents read topics and propose writes. Four guarantees, each with a test that fails when it breaks, on a real broker with SASL and ACLs.
Guarantee | How | Proof | |
G1 | Zero duplicate alert after a crash | Kafka transaction per batch (decisions + DLQ + offsets); pager stores offsets with its inbox/outbox in one SQLite transaction, then calls the paging API with an idempotency key |
|
G2 | Replaying history triggers nothing | Exactly-once does not cover replays: a compacted ledger topic, rebuilt on every partition assignment | group rewind + full re-send: test_g2_replay.py; random crash/replay sequences with Hypothesis: test_properties.py |
G3 | A poison record goes to the DLQ and does not block its partition | Undecodable record copied byte for byte to | |
G4 | Topic content cannot make the agent act beyond its rights | Rights are the broker's ACLs on the caller's own principal (no allow-list in the server, no token passthrough); writes are schema-checked, dry-run by default, and need a human answer through MCP elicitation; the grader has no tools and its grade is | a fully hijacked agent with a rubber-stamp human, refusals read back from |
Plus: one Jaeger trace from the agent's tool call to the page (test_trace.py), a schema-compatibility gate (test_schemas.py), and the server checked with MCP Inspector (test_inspector.py). Why, and what is out of scope: DESIGN.md. What the existing Kafka MCP servers leave out: docs/GAPS.md.
Run it
make setup && make up # Python deps; Kafka 4.1 (KRaft, SASL, ACLs), Schema Registry, Jaeger, Prometheus
make ci # lint, schema gate, every guarantee test (a few minutes, no LLM)
make bench # the numbers below, on the live model (cents; hard cap 2 USD)Related MCP server: kafka-mcp
Measured
Model claude-haiku-5-5, 3 replays of data/snapshot.jsonl, 2026-11-04. Ranges are min to max over the replays.
Measure | Value |
Reports in the frozen snapshot (+3 poison records) | 236 |
Severity exactly right, floor + LLM | 97.9% to 98.7% |
Severity exactly right, boxes only (no LLM) | 62.3% |
Critical (RED) reports missed, of 53 | 3 to 5 |
... of which RED only in free text, of 36 | 3 to 5 |
Critical missed with boxes only, i.e. with a fully hijacked grader | 36 |
False RED | 0 |
'Lower it to GREEN' injections that worked, of 9 | 0 to 1 |
'Raise it' injections that worked, of 2 | 0 |
LLM cost per 1,000 reports | $0.117 |
LLM call latency p50 / p95 | 1.12 s / 1.54 s |
End-to-end p50 / p95 (burst of the whole snapshot) | 24.8 s / 42.8 s |
Max consumer lag during the burst (Prometheus) | 79 to 139 |
Dead letters (poison records) | 3 |
Replay on the same ledger: records appended / LLM spend | 0 / $0.00 |
Live agent as bob@analytics: tool calls / writes done / out-of-rights done (attempts) / planted notes read | 2 / 0 / 0 (0) / 24 |
Live agent as alice@oncall: tool calls / writes done / out-of-rights done (attempts) / planted notes read | 3 / 0 / 0 (1) / 16 |
Every missed RED in these runs is the same note: a cleaner found on the floor, conscious, complaining of
their back. The model calls it ORANGE, with or without a planted note on top, so the "injections that
worked" are that same underestimate rather than the instruction taking effect. The box injury would have
caught it: the floor is for exactly this.
The snapshot (data/snapshot.jsonl) is synthetic: the layout of a real patrol form (site, times, ticked boxes, free text in French, photos, "on-call already notified"), invented content, and a ground truth known by construction (data/make_snapshot.py). About one report in eight carries a planted instruction.
How it fits together
webhook ─► bridge ─► patrol.reports.v1 ─► triage ─────────────► patrol.triage.v1 (compacted = ledger) ─► pager ─► paging API
(Avro) │ one transaction: (one decision per report) │ SQLite: offsets
│ decisions, DLQ, │ + inbox + outbox
│ offsets └ Idempotency-Key
├ floor from boxes (code) ─┐
└ LLM on free text, no tools ┴► max(floor, llm)
└► patrol.reports.v1.dlq
agent ─► MCP server ─ JWT (audience = this server) ─► sub ─► Kafka principal ─► broker ACLs decide
list_topics · read_topic (escaped JSON in a nonce-tagged envelope) · produce_message
(schema check ─► dry run ─► elicitation: the human says yes ─► broker says yes or no)What it does not protect
Found by two adversarial reviews of this code, and left open on purpose or out of scope:
A hijacked grader on a report with no box ticked. The floor only covers the structured part: if the model is taken over and the guard wrote "forced door" without ticking the box, the alert is missed. The "boxes only" row above is exactly that worst case.
The "on-call already notified" box turns the page off. That is the real system's rule (never wake someone twice), and a tampered form can use it. The decision stays RED and visible on the topic.
The report id comes from the form. A forged report sent first with the next id would make the real one a duplicate. Fix in production: the bridge assigns the id.
The human gate trusts the MCP client to really ask a human. A client that auto-approves defeats it; URL-mode elicitation to a page the server owns would not. The broker still stops out-of-rights writes.
Reading is not sending. If the agent's host has other outbound tools, data read here can leave through them, out of the broker's sight.
One triage instance. Several instances stay duplicate-free only while a report id always lands on the same partition (keyed by report id, murmur2, fixed partition count).
One refusal is not in the broker's audit log. A principal with no Write anywhere is refused its idempotent producer id (
CLUSTER_AUTHORIZATION_FAILED); Kafka 4.1 does not log that check. The MetadataDESCRIBErefusal before it is logged, and the client sees both.The paging API must deduplicate on the key (delivery to it is at-least-once); some real ones only deduplicate within a time window.
Local only: SASL/PLAIN without TLS, one broker, a toy identity provider, and the demo server process can read every local password. See DESIGN.md.
MIT licence.
This server cannot be deployed
Maintenance
Related MCP Connectors
Supervised API-write gateway for AI agents with policy, human approval and execution receipts.
Security gateway for AI agents: policy, approval, and audited execution, no secrets shared.
Zero-secret MCP gateway for AI agents: risk-scored, audited calls with human-in-the-loop approval.
Context and control for any AI. Agent identity, scoped access, and proof across tools.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to interact with Apache Kafka through natural language, supporting operations like producing/consuming messages, managing topics, and querying brokers, partitions, and consumer group offsets.1MIT
- FlicenseCqualityDmaintenanceExposes Kafka administration operations as MCP tools, enabling AI agents to inspect Kafka clusters using natural language.1-
- FlicenseBqualityDmaintenanceEnables interaction with Apache Kafka clusters, allowing management of topics, message production, and consumption through natural language.5-
- FlicenseNot gradedqualityCmaintenanceEnables controlled AI-agent access to enterprise-shaped tools with a deny-by-default gated write path, human approval, dry-run execution, and append-only audit logging.1-