# mcp-rag-service

A production-minded **Retrieval-Augmented Generation (RAG) service you can plug into any
LLM through the Model Context Protocol (MCP)** — with **PII redaction guardrails** and a
**retrieval-first evaluation harness**, packaged for **Docker and Kubernetes**.

I built this because I wanted a single, safe "answer questions over my documents" tool that
any MCP-aware assistant (Claude, an IDE agent, my own scripts) could call — without leaking
personal data and without me having to trust the answers blindly. So it answers only from
retrieved passages, cites its sources, redacts PII before anything touches the model, and
ships with an eval script that measures retrieval quality rather than vibes.

## Why it exists (the real problem)

An LLM on its own hallucinates and can't see your private documents. Wrapping retrieval
around it fixes both — but three things are usually missing from demo RAG:

1. **It isn't callable from other tools.** Here it's exposed as an **MCP server**, so any
   MCP client can use it as a first-class tool.
2. **It leaks PII.** Emails, phone numbers, IBANs and card-like numbers are **redacted
   before retrieval and before the prompt** (asymmetric-cost thinking: a redaction is cheap,
   an accidental leak into a model/log is expensive).
3. **Nobody measures it.** A `hit@k` + grounded-answer eval harness makes quality a number,
   not a feeling. Most "RAG hallucination" is really a retrieval miss, so I measure retrieval
   first.

## Architecture

```
MCP client (Claude / agent)
        │  tool: rag_query(question)
        ▼
   MCP server  (src/mcp_rag/server.py)
        │
   RAG pipeline (rag.py)
        │  1. redact PII        (guardrails.py)
        │  2. hybrid retrieve   (retriever.py: dense + BM25, score-fused)
        │  3. generate grounded, cited answer  (LLM, or extractive fallback)
        ▼
   {answer, citations[], redactions[]}
```

- **Hybrid retrieval:** dense embeddings (meaning) fused with BM25 (exact terms like IDs and
  names). Runs fully offline with a hashing embedder + pure-Python BM25 fallback, so there is
  **no API key required for development** — the paid LLM is used only for the final wording.
- **Guardrails:** `guardrails.py` redacts PII with well-tested patterns (and optionally
  Microsoft Presidio if installed).
- **Grounding:** answers are built only from retrieved chunks and carry `[n]` citations; if
  nothing relevant is retrieved it says so instead of guessing.

## Run it

```bash
pip install -r requirements.txt
python -m mcp_rag.ingest data/            # index your documents
python -m mcp_rag.server                  # start the MCP server (stdio)
```

Query it directly (no MCP client needed):

```bash
python -m mcp_rag.rag "What are the payment terms in the Nordwind contract?"
```

Evaluate retrieval quality:

```bash
python eval/run_eval.py                   # prints hit@k and grounded-answer rate
```

## Deploy (Docker + Kubernetes)

```bash
docker build -t mcp-rag-service -f deploy/Dockerfile .
kubectl apply -f deploy/k8s.yaml          # Deployment + Service, readiness/liveness probes
```

## Tech

Python · MCP (FastMCP) · FAISS / hashing-embedder fallback · BM25 · FastAPI (HTTP mode) ·
Presidio-optional PII redaction · pytest · Docker · Kubernetes.

## Status

Working core (retrieval, guardrails, MCP tool, eval) + container/K8s manifests. Corpus in
`data/` is a small sample; point `ingest` at your own documents to use it for real.