mcp-kafka-observer
Provides real-time observability into Apache Kafka clusters, enabling AI agents to monitor broker health, track consumer lag, and diagnose issues through natural language.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
mcp-kafka-observer
An MCP (Model Context Protocol) server that gives AI agents real-time observability into Apache Kafka clusters. Monitor broker health, track consumer lag, and diagnose issues — all through natural language.
Why?
Kafka monitoring typically requires juggling multiple dashboards. This MCP server lets any AI assistant (Claude, ChatGPT, Cursor, VS Code Copilot) query your Kafka cluster directly:
"Is my Kafka cluster healthy?"
"What's the consumer lag for payment-processor group?"
"Why is lag spiking on the orders topic?"
Related MCP server: kafka-mcp
Tools
Tool | Description |
| Cluster state: brokers, controller, under-replicated partitions |
| All topics with partition counts and replication factors |
| Detailed config and partition assignments for a topic |
| Per-partition lag for a consumer group |
| Automated root-cause analysis for lag issues |
| Cache hit/miss statistics for observability |
Resources
Resource URI | Description |
| High-level cluster summary |
Prompts
Prompt | Description |
| Step-by-step workflow for diagnosing consumer lag |
| Template for cluster capacity planning |
Quick Start
Prerequisites
Python 3.12+
Docker (for local Kafka)
uv package manager
Setup
git clone https://github.com/Rushi264/mcp-kafka-observer.git
cd mcp-kafka-observer
# Install dependencies
uv sync
# Start local Kafka
docker compose up -d
# Run tests
uv run pytest -vClaude Desktop Integration
Add to your claude_desktop_config.json:
{
"mcpServers": {
"kafka-observer": {
"command": "uv",
"args": [
"--directory", "/path/to/mcp-kafka-observer",
"run", "python", "-m", "mcp_kafka_observer.server"
],
"env": {
"KAFKA_BOOTSTRAP_SERVERS": "localhost:9092"
}
}
}
}Architecture
MCP Client (Claude / Cursor / VS Code Copilot)
│
│ MCP Protocol (stdio)
▼
mcp-kafka-observer
├── Tools (get_broker_health, get_consumer_lag, ...)
├── Resources (kafka://cluster/overview)
├── Prompts (investigate_lag, capacity_review)
├── TTL Cache (prevents thundering herd on admin API)
└── Analyzer (automated lag diagnosis)
│
│ confluent-kafka AdminClient
▼
Kafka ClusterTech Stack
Python 3.12 with async/await
MCP SDK (FastMCP) — official Anthropic SDK
confluent-kafka — production-grade Kafka client (librdkafka)
Pydantic — structured output validation
Docker Compose — local Kafka for development
Testing
# Unit tests (no Kafka needed)
uv run pytest tests/test_server.py -v
# Integration tests (needs Docker Kafka running)
docker compose up -d
uv run pytest tests/test_kafka_client.py -v
# All tests
uv run pytest -v
# Linter
uv run ruff check src/ tests/Configuration
Set via environment variables or .env file:
Variable | Default | Description |
|
| Kafka broker addresses |
| — | SASL auth mechanism (PLAIN, SCRAM-SHA-256) |
| — | SASL username |
| — | SASL password |
| — | Security protocol (SASL_SSL, SASL_PLAINTEXT) |
License
MIT
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
MCP server that lets your AI assistant build and deploy apps, hosting each one at a live URL.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceAn MCP server that enables interaction with Kafka clusters to manage topics, monitor consumer groups, and stream messages. It provides a comprehensive suite of tools for broker metadata inspection and local Kafka user management.MIT
- AlicenseBqualityBmaintenanceMCP server for Apache Kafka that allows LLM agents to inspect topics, consumer groups, and safely manage offsets (reset, rewind).1914Apache 2.0
- AlicenseNot gradedqualityAmaintenanceAn MCP server that enables AI assistants to safely interact with Apache Kafka clusters, providing tools for topic management, message operations, consumer groups, and cluster information.3MIT
- AlicenseAqualityAmaintenanceAn MCP server that gives AI agents full visibility and control over your Dagster instance, enabling autonomous monitoring, diagnosis, and remediation of data pipelines.1710,101 PyPI18MIT