agentic-rag-mcp

A weekend build to get the three buzzwords straight in my own head by actually wiring them up: agentic RAG, MCP, and a small multi-agent pipeline, all over the same tiny document set. It's a working sketch, not a framework, and not production code — the point was to see the moving parts and where each idea earns its keep (and where it doesn't).

The knowledge base is five short docs about a fictional library called Halyard (docs/). They're written so a couple of questions genuinely need more than one lookup — which is the whole reason "agentic" retrieval is interesting here.

What runs without an API key, and what needs one

The LLM pieces use Claude (claude-opus-4-8, adaptive thinking, prompt caching on the system/tools prefix). Without a key, agentic_rag.py falls back to a naive single-shot retrieval baseline and multi_agent.py runs stub roles, so you can still watch the control flow.

Related MCP server: Hybrid RAG MCP Server

Retrieval (knowledge_base.py)

Nothing clever: fixed-size character chunks with overlap, MiniLM embeddings, a FAISS flat inner-product index. No reranker, no hybrid BM25, no semantic chunking. Those are the obvious quality upgrades, but they'd be beside the point — this repo is about the orchestration on top, so the retriever is intentionally the dumb part.

Agentic RAG (agentic_rag.py)

Naive RAG retrieves once and answers. That breaks on questions like "how do I authenticate SqlSource after 3.0, and is the old way still supported?" — the answer is split across the migration guide and the changelog, and a single top-k search usually grabs one and misses the other.

Agentic RAG hands the model a kb_search tool and lets it decide: search, read the results, notice the second half of the question isn't covered, search again, then answer with citations. Same FAISS backend; the model just does multi-hop lookups itself. agentic_rag.py prints each query it chooses so you can see the hops.

The trade-off is real and worth stating: the agent costs several model round-trips and can occasionally wander or over-search. For single-fact questions naive RAG is cheaper and just as good. The agent earns its cost only when one lookup genuinely isn't enough.

MCP (mcp_server.py, mcp_client_demo.py)

The agents above could just import knowledge_base. MCP is about not doing that. Model Context Protocol is a standard way for a server to advertise tools and for any client — Claude Desktop, an IDE, your own agent — to discover and call them without custom glue. Run one retrieval server, point many clients at it.

mcp_server.py is that server (built with the official SDK's FastMCP), serving three tools over stdio. mcp_client_demo.py is the easiest way to see it work: it spawns the server as a subprocess, does the MCP handshake, lists the tools, and calls them — all locally, no key:

python mcp_client_demo.py

To use the same server from Claude Desktop, add it to that app's claude_desktop_config.json:

{
  "mcpServers": {
    "halyard-kb": {
      "command": "python",
      "args": ["C:/Users/you/.../agentic-rag-mcp/mcp_server.py"]
    }
  }
}

Then Claude can call kb_search directly. The point: the retrieval logic lives in one place behind a stable interface, and the consumer doesn't know or care that it's FAISS underneath.

Multi-agent (multi_agent.py)

Same task, decomposed into three focused roles, orchestrated in plain Python:

question -> RETRIEVER (has the search tool) -> evidence
         -> SYNTHESIZER (no tools)          -> draft answer
         -> VERIFIER (structured output)    -> supported? / issues
              supported -> return
              not       -> feed issues back to the synthesizer, retry (<= N)

That's two standard patterns — orchestrator/workers and evaluator/optimizer (the verify-and-revise loop). The flow is ordinary code; I'm deliberately not letting agents spawn each other freely. For a task this well-shaped, a fixed workflow is more predictable, cheaper, and far easier to debug than an autonomous swarm, and you can read exactly what happened. Autonomous multi-agent is worth it when the task genuinely can't be scripted up front — this one can, so it's a workflow with LLM steps, and I think that's the honest default for most "agent" problems.

The three roles are just functions, so the LLM version and an offline stub version share the same orchestrate() loop.

Running it

pip install -r requirements.txt

python knowledge_base.py     # retrieval sanity check
python mcp_client_demo.py    # MCP server + client, end to end, offline

# with a key, the LLM versions:
export ANTHROPIC_API_KEY=...        # PowerShell: $env:ANTHROPIC_API_KEY="..."
python agentic_rag.py
python multi_agent.py

First run downloads the ~90 MB embedding model. Requirements: Python 3.9+, numpy, faiss-cpu, sentence-transformers, mcp, and (for the LLM steps) anthropic + pydantic.

Caveats / what I'd do differently for real work

• It's five toy docs. Retrieval quality, chunking, and reranking would matter a lot more on a real corpus and are barely exercised here.

• I never measured anything. To make claims about "agentic beats naive" I'd build a small eval set of multi-hop questions with reference answers and actually score them, including tokens and round-trips, not just eyeball the traces.

• The verifier is an LLM judging against retrieved text; it can be wrong and tends to be lenient. I'd calibrate it against a few human-checked answers.

• The agent loop has no budget cap beyond max_turns; a real one wants a token budget and a cost ceiling.

• In a real deployment the agents would call the KB through the MCP server (the SDK has helpers to bridge MCP tools into the tool loop); here they call the KB directly for simplicity, and MCP is shown as its own slice.

Muhammad Farooqi · https://github.com/mqfarooqi1