🌊🔍 Lenses MCP Server for Apache Kafka 🔎🌊

This is the Lenses MCP (Model Context Protocol) server for Apache Kafka. Lenses offers a developer experience solution for engineers building real-time applications connected to Kafka. It's built for the enterprise and backed by a powerful IAM and governance model.

With Lenses, you can find, explore, transform, integrate and replicate data across a multi-Kafka and vendor estate. Now, all this power is accessible through your AI tools and AI Agents via MCP, bringing real-time context into your agentic engineering workflows.

The quickest way to try the MCP server is with the free Lenses Community Edition, which runs Lenses MCP Server as a remote MCP server and comes with a pre-configured single broker Kafka cluster with demo data, ideal for local development or evaluation (steps here).

Table of Contents

• 1. Install uv and Python

• 2. Configure Environment Variables

• 3. OAuth 2.1 Authentication (Recommended)

• 4. Lenses API Key (Fallback)

• 5. Running the Server Locally

• 6. Running with Docker

• 7. Optional Context7 MCP Server

• Appendix: OAuth Flow Details

1. Install uv and Python

We use uv for dependency management and project setup. If you don't have uv installed, follow the official installation guide.

This project has been built using Python 3.12 and to make sure Python is correctly installed, run the following command to check the version.

uv run python --version

2. Configure Environment Variables

Copy the example environment file and configure it based on your authentication method:

cp .env.example .env

Required variables depend on your authentication choice:

• For OAuth (recommended): LENSES_URL and MCP_ADVERTISED_URL

• For API Key (fallback): LENSES_URL and LENSES_API_KEY

3. OAuth 2.1 Authentication (Recommended)

OAuth 2.1 is the recommended authentication method for all Lenses MCP deployments. It provides secure, scope-based authorization without sharing static API keys.

How it works

OAuth 2.1 uses bearer tokens that are validated via RFC 7662 Token Introspection. The flow involves three participants:

1. MCP Client — Your AI tool (Claude, Cursor, etc.)

2. Authorization Server — Lenses HQ at LENSES_ADVERTISED_URL

3. MCP Server — This server (the resource server)

When you connect, the client automatically:

1. Discovers OAuth metadata from this server (/.well-known/oauth-protected-resource/mcp)

2. Registers itself with the authorization server

3. Initiates OAuth authorization (with PKCE) and gets an access token

4. Uses the token to authenticate requests to this MCP server

This server then validates the token with Lenses HQ before allowing access to Kafka resources.

Simple setup

To use OAuth, you should set OAUTH_ENABLED to true, then you only need to set two environment variables:

OAUTH_ENABLED=true
LENSES_URL=https://lenses.example.com
MCP_ADVERTISED_URL=http://localhost:8000

• LENSES_URL — Your Lenses instance (used internally and as the OAuth authorization server)

• MCP_ADVERTISED_URL — The public URL where this MCP server is reachable by clients

TRANSPORT automatically defaults to http when MCP_ADVERTISED_URL is set.

Advanced: split-plane deployments

If the MCP server reaches Lenses on an internal address but clients reach it on a public URL:

OAUTH_ENABLED=true
LENSES_URL=http://lenses-hq.internal:9991
LENSES_ADVERTISED_URL=https://lenses.example.com
MCP_ADVERTISED_URL=https://mcp.example.com

Authorization scopes

The server advertises three scopes:

When you authenticate, you'll be prompted to grant these scopes. Your token will only grant the scopes you select.

Lenses HQ configuration

Lenses HQ must support OAuth 2.0 and token introspection. Ensure your Lenses HQ config includes:

oauth2:
  authorizationServer:
    unauthenticatedIntrospection: true

This allows the MCP server to validate tokens without client credentials.

4. Lenses API Key (Fallback)

For backward compatibility and testing, you can use a static API key instead of OAuth. This is not recommended for production but may be useful for local development or legacy systems.

Create a Lenses API key by provisioning an IAM Service Account in Lenses. Add the API key to .env:

LENSES_URL=https://lenses.example.com
LENSES_API_KEY=<YOUR_LENSES_API_KEY>

When using API key authentication, TRANSPORT defaults to stdio (local only) unless you explicitly set MCP_ADVERTISED_URL.

5. Running the Server Locally

First, install dependencies:

uv sync

With OAuth (Recommended)

Run with stdio transport (for local AI tools):

OAUTH_ENABLED=true \
LENSES_URL=https://lenses.example.com \
MCP_ADVERTISED_URL=http://localhost:8000 \
uv run src/lenses_mcp/server.py

Or run with HTTP transport (for remote clients):

OAUTH_ENABLED=true \
LENSES_URL=https://lenses.example.com \
MCP_ADVERTISED_URL=http://localhost:8000 \
uv run fastmcp run src/lenses_mcp/server.py --transport=http --port=8000

To configure in Claude Desktop, Cursor, or similar tools:

{
  "mcpServers": {
    "Lenses": {
      "command": "uv",
      "args": [
        "run",
        "--project", "<ABSOLUTE_PATH_TO_THIS_REPO>",
        "--with", "fastmcp",
        "fastmcp",
        "run",
        "<ABSOLUTE_PATH_TO_THIS_REPO>/src/lenses_mcp/server.py"
      ],
      "env": {
        "OAUTH_ENABLED": "true",
        "LENSES_URL": "https://lenses.example.com",
        "MCP_ADVERTISED_URL": "http://localhost:8000"
      },
      "transport": "stdio"
    }
  }
}

With API Key (Legacy)

Using a static API key:

LENSES_URL=https://lenses.example.com \
LENSES_API_KEY=<YOUR_LENSES_API_KEY> \
uv run src/lenses_mcp/server.py

Or with HTTP transport:

LENSES_URL=https://lenses.example.com \
LENSES_API_KEY=<YOUR_LENSES_API_KEY> \
uv run fastmcp run src/lenses_mcp/server.py --transport=http --port=8000

To configure in Claude Desktop, Cursor, or similar tools:

{
  "mcpServers": {
    "Lenses.io": {
      "command": "uv",
      "args": [
        "run",
        "--project", "<ABSOLUTE_PATH_TO_THIS_REPO>",
        "--with", "fastmcp",
        "fastmcp",
        "run",
        "<ABSOLUTE_PATH_TO_THIS_REPO>/src/lenses_mcp/server.py"
      ],
      "env": {
        "LENSES_URL": "https://lenses.example.com",
        "LENSES_API_KEY": "<YOUR_LENSES_API_KEY>"
      },
      "transport": "stdio"
    }
  }
}

Note: Some clients may require the absolute path to uv in the command.

6. Running with Docker

The Lenses MCP server is available as a Docker image at lensesio/mcp. You can run it with OAuth (recommended) or API key authentication.

With OAuth (Recommended)

Stdio transport (for local AI tools):

docker run --rm -it \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=http://localhost:8000 \
  lensesio/mcp

HTTP transport (for remote clients, listens on http://0.0.0.0:8000/mcp):

docker run --rm -it -p 8000:8000 \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=http://localhost:8000 \
  -e TRANSPORT=http \
  lensesio/mcp

For split-plane deployments where the MCP server reaches Lenses internally but clients use a public URL:

docker run --rm -it -p 8000:8000 \
  -e OAUTH_ENABLED=true \
  -e LENSES_URL=http://lenses-hq.internal:9991 \
  -e LENSES_ADVERTISED_URL=https://lenses.example.com \
  -e MCP_ADVERTISED_URL=https://mcp.example.com \
  -e TRANSPORT=http \
  lensesio/mcp

With API Key (Legacy)

Stdio transport (for local AI tools):

docker run --rm -it \
  -e LENSES_API_KEY=<YOUR_API_KEY> \
  -e LENSES_URL=https://lenses.example.com \
  lensesio/mcp

HTTP transport (for remote clients, listens on http://0.0.0.0:8000/mcp):

docker run --rm -it -p 8000:8000 \
  -e LENSES_API_KEY=<YOUR_API_KEY> \
  -e LENSES_URL=https://lenses.example.com \
  -e TRANSPORT=http \
  lensesio/mcp

Environment Variables Reference

Legacy environment variables (for backward compatibility):

• LENSES_API_HTTP_URL, LENSES_API_HTTP_PORT

• LENSES_API_WEBSOCKET_URL, LENSES_API_WEBSOCKET_PORT

These are automatically derived from LENSES_URL but can be explicitly set to override.

Transport Endpoints

• stdio: Standard input/output (no network endpoint)

• http: HTTP endpoint at /mcp

• sse: Server-Sent Events endpoint at /sse

Building the Docker Image Locally

To build the Docker image locally:

docker build -t lensesio/mcp .

7. Optional Context7 MCP Server

Lenses documentation is available on Context7. It is optional but highly recommended to use the Context7 MCP Server and adjust your prompts with use context7 to ensure the documentation available to the LLM is up to date.

Appendix: OAuth Flow Details

Token validation sequence

The MCP server validates bearer tokens using the following sequence:

1. Protected Resource Metadata (RFC 9728) — RemoteAuthProvider serves /.well-known/oauth-protected-resource/mcp so clients can discover which authorization server to use and what scopes are available.

2. Auto-Discovery — On the first incoming request, the DiscoveryTokenVerifier lazily fetches {LENSES_ADVERTISED_URL}/.well-known/oauth-authorization-server to discover the introspection_endpoint. The endpoint URL can also be set explicitly via INTROSPECTION_URL.

3. Token Introspection (RFC 7662) — For each incoming bearer token, the verifier POSTs to the introspection endpoint (/oauth2/introspect) without client authentication. The authorization server responds with:

   • active — whether the token is valid

   • scope — granted scopes (e.g. read write)

   • client_id — the token's owner

   • exp — expiration timestamp

   Inactive or expired tokens are rejected before reaching the Lenses API.

4. Token Forwarding — Valid tokens are forwarded to the Lenses API via Authorization: Bearer <token> so Lenses can perform its own authorization checks.

Authorization scopes

The server advertises three scopes in its protected-resource metadata:

Scopes are not enforced globally at the introspection level — a token with any subset of these scopes is accepted. Per-tool scope enforcement can be added using FastMCP's require_scopes decorator.

Configuration and Requirements

In a simple deployment, only two environment variables are required:

LENSES_URL=https://lenses.example.com
MCP_ADVERTISED_URL=http://localhost:8000

For split-plane deployments where the MCP server reaches Lenses on an internal address but clients use a public URL, set:

LENSES_URL=http://lenses-hq.internal:9991
LENSES_ADVERTISED_URL=https://lenses.example.com
MCP_ADVERTISED_URL=https://mcp.example.com

Lenses HQ must support:

• OAuth 2.0 Authorization Server Metadata (RFC 8414) at /.well-known/oauth-authorization-server

• Token Introspection (RFC 7662) at the introspection_endpoint, with client authentication disabled

• PKCE with S256 (RFC 7636) for client authorization flows

The MCP server does not send client credentials when introspecting. Lenses HQ must be configured with:

oauth2:
  authorizationServer:
    unauthenticatedIntrospection: true

Without this setting, every bearer token will be rejected as invalid.