Provides publish and consume functionalities for Kafka topics, allowing messages to be sent to and retrieved from Kafka streams. Messages consumed cannot be read again using the same group ID.
Supports environment variable configuration through .env files for Kafka connection settings and tool descriptions.
Utilizes Pydantic for settings management and validation of configuration values.
Implemented in Python with support for Python 3.8+ environments.
Kafka MCP Server
A Message Context Protocol (MCP) server that integrates with Apache Kafka to provide publish and consume functionalities for LLM and Agentic applications.
Overview
This project implements a server that allows AI models to interact with Kafka topics through a standardized interface. It supports:
- Publishing messages to Kafka topics
- Consuming messages from Kafka topics
Prerequisites
- Python 3.8+
- Apache Kafka instance
- Python dependencies (see Installation section)
Installation
- Clone the repository:
- Create a virtual environment and activate it:
- Install the required dependencies:If no requirements.txt exists, install the following packages:
Configuration
Create a .env
file in the project root with the following variables:
Usage
Running the Server
You can run the server using the provided main.py
script:
Available transport options:
stdio
: Standard input/output (default)sse
: Server-Sent Events
Integrating with Claude Desktop
To use this Kafka MCP server with Claude Desktop, add the following configuration to your Claude Desktop configuration file:
Replace <PATH TO PROJECTS>
with the absolute path to your project directory.
Project Structure
main.py
: Entry point for the applicationkafka.py
: Kafka connector implementationserver.py
: MCP server implementation with tools for Kafka interactionsettings.py
: Configuration management using Pydantic
Available Tools
kafka-publish
Publishes information to the configured Kafka topic.
kafka-consume
consume information from the configured Kafka topic.
- Note: once a message is read from the topic it can not be read again using the same groupid
Create-Topic
Creates a new Kafka topic with specified parameters.
- Options:
--topic
Name of the topic to create--partitions
Number of partitions to allocate--replication-factor
Replication factor across brokers--config
(optional) Topic-level configuration overrides (e.g.,retention.ms=604800000
)
Delete-Topic
Deletes an existing Kafka topic.
- Options:
--topic
Name of the topic to delete--timeout
(optional) Time to wait for deletion to complete
List-Topics
Lists all topics in the cluster (or filtered by pattern).
- Options:
--bootstrap-server
Broker address--pattern
(optional) Regular expression to filter topic names--exclude-internal
(optional) Exclude internal topics (default: true)
Topic-Configuration
Displays or alters configuration for one or more topics.
- Options:
--describe
Show current configs for a topic--alter
Modify configs (e.g.,--add-config retention.ms=86400000,--delete-config cleanup.policy
)--topic
Name of the topic
Topic-Metadata
Retrieves metadata about a topic or the cluster.
- Options:
--topic
(If provided) Fetch metadata only for this topic--bootstrap-server
Broker address--include-offline
(optional) Include brokers or partitions that are offline
This server cannot be installed
remote-capable server
The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.
Enables AI models to publish and consume messages from Apache Kafka topics through a standardized interface, making it easy to integrate Kafka messaging with LLM and agent applications.
Related Resources
Related MCP Servers
- AsecurityAlicenseAqualityEnables AI models to interact with messages from various messaging platforms (Mobile, Mail, WhatsApp, LinkedIn, Slack, Twitter, Telegram, Instagram, Messenger) through a standardized interface.Last updated -311PythonMIT License
- -securityFlicense-qualityProvides AI agents with comprehensive Twitter functionality through the Model Context Protocol standard, enabling reading tweets, posting content, managing interactions, and accessing timeline data with robust error handling.Last updated -411JavaScript
- AsecurityAlicenseAqualityA server implementing Model Context Protocol that enables AI assistants to interact with Slack API through a standardized interface, providing tools for messaging, channel management, user information retrieval, and more.Last updated -980JavaScriptApache 2.0
- -securityFlicense-qualityAn implementation of the Model Context Protocol that connects AI agents to Kakao Official Accounts, allowing users to send various message templates through the Kakao Developers API.Last updated -6Python