mcp-kafka-observer
Provides real-time observability into Apache Kafka clusters, enabling AI agents to monitor broker health, track consumer lag, and diagnose issues through natural language.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-kafka-observerWhat's the consumer lag for payment-processor group?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-kafka-observer
An MCP (Model Context Protocol) server that gives AI agents real-time observability into Apache Kafka clusters. Monitor broker health, track consumer lag, and diagnose issues — all through natural language.
Why?
Kafka monitoring typically requires juggling multiple dashboards. This MCP server lets any AI assistant (Claude, ChatGPT, Cursor, VS Code Copilot) query your Kafka cluster directly:
"Is my Kafka cluster healthy?"
"What's the consumer lag for payment-processor group?"
"Why is lag spiking on the orders topic?"
Related MCP server: Kafka MCP Server
Tools
Tool | Description |
| Cluster state: brokers, controller, under-replicated partitions |
| All topics with partition counts and replication factors |
| Detailed config and partition assignments for a topic |
| Per-partition lag for a consumer group |
| Automated root-cause analysis for lag issues |
| Cache hit/miss statistics for observability |
Resources
Resource URI | Description |
| High-level cluster summary |
Prompts
Prompt | Description |
| Step-by-step workflow for diagnosing consumer lag |
| Template for cluster capacity planning |
Quick Start
Prerequisites
Python 3.12+
Docker (for local Kafka)
uv package manager
Setup
git clone https://github.com/Rushi264/mcp-kafka-observer.git
cd mcp-kafka-observer
# Install dependencies
uv sync
# Start local Kafka
docker compose up -d
# Run tests
uv run pytest -vClaude Desktop Integration
Add to your claude_desktop_config.json:
{
"mcpServers": {
"kafka-observer": {
"command": "uv",
"args": [
"--directory", "/path/to/mcp-kafka-observer",
"run", "python", "-m", "mcp_kafka_observer.server"
],
"env": {
"KAFKA_BOOTSTRAP_SERVERS": "localhost:9092"
}
}
}
}Architecture
MCP Client (Claude / Cursor / VS Code Copilot)
│
│ MCP Protocol (stdio)
▼
mcp-kafka-observer
├── Tools (get_broker_health, get_consumer_lag, ...)
├── Resources (kafka://cluster/overview)
├── Prompts (investigate_lag, capacity_review)
├── TTL Cache (prevents thundering herd on admin API)
└── Analyzer (automated lag diagnosis)
│
│ confluent-kafka AdminClient
▼
Kafka ClusterTech Stack
Python 3.12 with async/await
MCP SDK (FastMCP) — official Anthropic SDK
confluent-kafka — production-grade Kafka client (librdkafka)
Pydantic — structured output validation
Docker Compose — local Kafka for development
Testing
# Unit tests (no Kafka needed)
uv run pytest tests/test_server.py -v
# Integration tests (needs Docker Kafka running)
docker compose up -d
uv run pytest tests/test_kafka_client.py -v
# All tests
uv run pytest -v
# Linter
uv run ruff check src/ tests/Configuration
Set via environment variables or .env file:
Variable | Default | Description |
|
| Kafka broker addresses |
| — | SASL auth mechanism (PLAIN, SCRAM-SHA-256) |
| — | SASL username |
| — | SASL password |
| — | Security protocol (SASL_SSL, SASL_PLAINTEXT) |
License
MIT
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Rushi264/mcp-kafka-observer'
If you have feedback or need assistance with the MCP directory API, please join our Discord server