OTel Config Copilot

An AI agent that helps engineers understand and validate OpenTelemetry Collector YAML. Collector configs are error-prone because processor order is semantically meaningful and YAML will not enforce it — memory_limiter must come first, batch should come after sampling, and a Collector will happily start with those reversed.

Three Python modules, one optional UI:

api.py + web/ wrap the agent in FastAPI and a React chat UI.

A one-page walkthrough of the boxes and arrows is in ARCHITECTURE.md.

Architecture

  engineer
     │
     ├─ python agent.py          (CLI)
     ├─ React chat → FastAPI     (/api/chat)
     └─ Cursor / Claude Desktop  (MCP stdio or http://127.0.0.1:8000/mcp)
              │
              ▼
        agent.py  while True:
          generate_content(tools=…)
          function_call present?
              │ yes                    │ no
              ▼                        ▼
        TOOL_FUNCTIONS[name]        return text
        (search / validate / explain)
              │
              ▼
           rag.py  ←  sklearn TF-IDF corpus
                      + COMPONENT_INDEX exact hits

The agent does not speak MCP on the hot path. It imports the same Python functions the MCP server registers. MCP is a discovery/transport layer, not a second implementation.

Why MCP

MCP standardizes tool discovery. Any MCP-compatible client can list these tools, read their docstrings, and call them without custom integration code — no one has to copy our JSON schemas into Cursor, Claude Desktop, or a future host. The alternative is a proprietary function-calling payload per product.

That is also why the MCP tool docstrings are written for a model, not a human: the host forwards them as the tool description.

SDK pin: mcp==2.1.1. v1 called the high-level class FastMCP (mcp.server.fastmcp). v2 renamed it MCPServer and moved it to mcp.server.mcpserver. There is no alias; importing FastMCP on 2.x fails at startup.

Related MCP server: agentvet-mcp

Hybrid retrieval (the live bug)

search_docs("memory_limiter") originally returned the 5-token cheat sheet instead of the limiter guide.

Cause: sklearn's TfidfVectorizer L2-normalizes each document vector. For a one-token query, cosine similarity collapses to "what fraction of this document is that term?". A short doc that mentions memory_limiter outranks a long doc that explains it.

Fix: COMPONENT_INDEX maps known names (otlp, batch, memory_limiter, tail_sampling, debug) to doc ids and short-circuits TF-IDF. Production RAG would use embeddings for semantic rather than lexical matching; this overlay is what you ship when the corpus is tiny and must run offline.

The regression is locked in tests/test_rag.py:

• tfidf_search("memory_limiter") still ranks cheat_sheet first (the bug)

• search_docs("memory_limiter") returns the limiter doc with source=exact (the fix)

Interview version of this: "I found single-word lookups were returning the wrong doc, traced it to TF-IDF's length normalization, and added exact-match routing."

Provider

Google Gemini via the google-genai SDK (==2.22.0). Default model: gemini-2.5-flash — free tier (10 RPM / 250 RPD / 250K TPM), strong function-calling support.

Get a free API key at aistudio.google.com/app/apikey. Override the model with GEMINI_MODEL in .env.

Why manual tool dispatch? The SDK's Automatic Function Calling (AFC) silently persists state across requests when history contains prior function_call parts (see #1818). We disable AFC and handle the loop explicitly — same while True structure as before, just against client.models.generate_content().

Setup

Python 3.10+. Create a venv and install:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env   # add your GEMINI_API_KEY

The knowledge base and YAML validator do not need an API key. The agent loop does.

pytest tests/ -q

How to run

CLI agent (conversation history across turns)

python agent.py

Paste examples/invalid-pipeline.yaml and ask it to validate. Watch stdout: each tool call is printed as → name({args}) before the model answers, so you can see chaining (validate → explain → answer).

Type exit or Ctrl-D to quit.

MCP server (stdio)

python mcp_server.py

This blocks on stdin. Point a host at it. Cursor example (~/.cursor/mcp.json):

{
  "mcpServers": {
    "otel-config-copilot": {
      "command": "python",
      "args": ["/absolute/path/to/mcp_server.py"]
    }
  }
}

Or the Inspector: mcp dev mcp_server.py.

HTTP: FastAPI + React chat

# terminal 1 — API, MCP HTTP mount, agent
uvicorn api:app --reload --port 8000

# terminal 2 — Vite dev server, proxies /api → :8000
npm --prefix web install
npm --prefix web run dev

Open http://127.0.0.1:5173. MCP hosts can also POST to http://127.0.0.1:8000/mcp.

To serve the built UI from FastAPI instead of Vite:

npm --prefix web run build
uvicorn api:app --port 8000
# UI at http://127.0.0.1:8000/

Tools

Validator rules, by design, are stricter than otelcol validate:

1. Document is parseable YAML and a mapping

2. Top-level receivers, processors, exporters, service all present

3. In every pipeline that lists memory_limiter (or memory_limiter/<instance>), it is index 0

otelcol validate will not catch (3). That is the whole point.

Knowledge base coverage

Concept docs: receivers, processors, exporters, pipelines, validation, fleet management (OpAMP).

Component docs: otlp, batch, memory_limiter, tail_sampling, debug.

Plus a deliberately tiny cheat_sheet that exists so the TF-IDF bug is reproducible in tests.