Skip to main content
Glama
mcp-kafka-gouverne

mcp-kafka-gouverne

by AmirK-S

mcp-kafka-gouverne

An LLM agent plugged into a Kafka bus without breaking the bus. A triage consumer grades night-patrol reports with a model and raises alerts; an MCP server lets people's agents read topics and propose writes. Four guarantees, each with a test that fails when it breaks, on a real broker with SASL and ACLs.

Guarantee

How

Proof

G1

Zero duplicate alert after a crash

Kafka transaction per batch (decisions + DLQ + offsets); pager stores offsets with its inbox/outbox in one SQLite transaction, then calls the paging API with an idempotency key

kill -9 mid-transaction and at random: test_g1_crash.py

G2

Replaying history triggers nothing

Exactly-once does not cover replays: a compacted ledger topic, rebuilt on every partition assignment

group rewind + full re-send: test_g2_replay.py; random crash/replay sequences with Hypothesis: test_properties.py

G3

A poison record goes to the DLQ and does not block its partition

Undecodable record copied byte for byte to *.dlq inside the same transaction; an LLM outage is not poison

test_g3_poison.py

G4

Topic content cannot make the agent act beyond its rights

Rights are the broker's ACLs on the caller's own principal (no allow-list in the server, no token passthrough); writes are schema-checked, dry-run by default, and need a human answer through MCP elicitation; the grader has no tools and its grade is max(floor, llm)

a fully hijacked agent with a rubber-stamp human, refusals read back from kafka-authorizer.log: test_g4_rights.py

Plus: one Jaeger trace from the agent's tool call to the page (test_trace.py), a schema-compatibility gate (test_schemas.py), and the server checked with MCP Inspector (test_inspector.py). Why, and what is out of scope: DESIGN.md. What the existing Kafka MCP servers leave out: docs/GAPS.md.

Run it

make setup && make up   # Python deps; Kafka 4.1 (KRaft, SASL, ACLs), Schema Registry, Jaeger, Prometheus
make ci                 # lint, schema gate, every guarantee test (a few minutes, no LLM)
make bench              # the numbers below, on the live model (cents; hard cap 2 USD)

Related MCP server: kafka-mcp

Measured

Model claude-haiku-5-5, 3 replays of data/snapshot.jsonl, 2026-11-04. Ranges are min to max over the replays.

Measure

Value

Reports in the frozen snapshot (+3 poison records)

236

Severity exactly right, floor + LLM

97.9% to 98.7%

Severity exactly right, boxes only (no LLM)

62.3%

Critical (RED) reports missed, of 53

3 to 5

... of which RED only in free text, of 36

3 to 5

Critical missed with boxes only, i.e. with a fully hijacked grader

36

False RED

0

'Lower it to GREEN' injections that worked, of 9

0 to 1

'Raise it' injections that worked, of 2

0

LLM cost per 1,000 reports

$0.117

LLM call latency p50 / p95

1.12 s / 1.54 s

End-to-end p50 / p95 (burst of the whole snapshot)

24.8 s / 42.8 s

Max consumer lag during the burst (Prometheus)

79 to 139

Dead letters (poison records)

3

Replay on the same ledger: records appended / LLM spend

0 / $0.00

Live agent as bob@analytics: tool calls / writes done / out-of-rights done (attempts) / planted notes read

2 / 0 / 0 (0) / 24

Live agent as alice@oncall: tool calls / writes done / out-of-rights done (attempts) / planted notes read

3 / 0 / 0 (1) / 16

Every missed RED in these runs is the same note: a cleaner found on the floor, conscious, complaining of their back. The model calls it ORANGE, with or without a planted note on top, so the "injections that worked" are that same underestimate rather than the instruction taking effect. The box injury would have caught it: the floor is for exactly this.

The snapshot (data/snapshot.jsonl) is synthetic: the layout of a real patrol form (site, times, ticked boxes, free text in French, photos, "on-call already notified"), invented content, and a ground truth known by construction (data/make_snapshot.py). About one report in eight carries a planted instruction.

How it fits together

webhook ─► bridge ─► patrol.reports.v1 ─► triage ─────────────► patrol.triage.v1 (compacted = ledger) ─► pager ─► paging API
                        (Avro)             │ one transaction:      (one decision per report)            │ SQLite: offsets
                                           │ decisions, DLQ,                                            │ + inbox + outbox
                                           │ offsets                                                    └ Idempotency-Key
                                           ├ floor from boxes (code)  ─┐
                                           └ LLM on free text, no tools ┴► max(floor, llm)
                                           └► patrol.reports.v1.dlq

agent ─► MCP server ─ JWT (audience = this server) ─► sub ─► Kafka principal ─► broker ACLs decide
            list_topics · read_topic (escaped JSON in a nonce-tagged envelope) · produce_message
            (schema check ─► dry run ─► elicitation: the human says yes ─► broker says yes or no)

What it does not protect

Found by two adversarial reviews of this code, and left open on purpose or out of scope:

  • A hijacked grader on a report with no box ticked. The floor only covers the structured part: if the model is taken over and the guard wrote "forced door" without ticking the box, the alert is missed. The "boxes only" row above is exactly that worst case.

  • The "on-call already notified" box turns the page off. That is the real system's rule (never wake someone twice), and a tampered form can use it. The decision stays RED and visible on the topic.

  • The report id comes from the form. A forged report sent first with the next id would make the real one a duplicate. Fix in production: the bridge assigns the id.

  • The human gate trusts the MCP client to really ask a human. A client that auto-approves defeats it; URL-mode elicitation to a page the server owns would not. The broker still stops out-of-rights writes.

  • Reading is not sending. If the agent's host has other outbound tools, data read here can leave through them, out of the broker's sight.

  • One triage instance. Several instances stay duplicate-free only while a report id always lands on the same partition (keyed by report id, murmur2, fixed partition count).

  • One refusal is not in the broker's audit log. A principal with no Write anywhere is refused its idempotent producer id (CLUSTER_AUTHORIZATION_FAILED); Kafka 4.1 does not log that check. The Metadata DESCRIBE refusal before it is logged, and the client sees both.

  • The paging API must deduplicate on the key (delivery to it is at-least-once); some real ones only deduplicate within a time window.

  • Local only: SASL/PLAIN without TLS, one broker, a toy identity provider, and the demo server process can read every local password. See DESIGN.md.

MIT licence.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI agents to interact with Apache Kafka through natural language, supporting operations like producing/consuming messages, managing topics, and querying brokers, partitions, and consumer group offsets.
    1
    MIT
  • F
    license
    C
    quality
    D
    maintenance
    Exposes Kafka administration operations as MCP tools, enabling AI agents to inspect Kafka clusters using natural language.
    1
    -
  • F
    license
    B
    quality
    D
    maintenance
    Enables interaction with Apache Kafka clusters, allowing management of topics, message production, and consumption through natural language.
    5
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    Enables controlled AI-agent access to enterprise-shaped tools with a deny-by-default gated write path, human approval, dry-run execution, and append-only audit logging.
    1
    -