# RAG + MCP Agent Demo

A compact, **dependency-free** reference implementation of the core pieces behind a production LLM/agent stack:

- a **RAG pipeline** (embed → store → retrieve → assemble context),
- an **MCP server** exposing read-only tools over stdio (JSON-RPC 2.0),
- a **Text-to-SQL safety gate** that validates generated SQL *before* execution,
- an **offline evaluation harness** (retrieval recall@k + SQL-guard accuracy),
- and a small **agent loop** that routes between tools.

> This is a sanitized, self-contained demo. It contains **no** employer code or data.

## Why this exists

Most "RAG demos" show a notebook that retrieves a chunk. Production systems need the boring parts: a swappable vector store, a hard safety boundary in front of the database, and an evaluation harness that fails CI when quality regresses. This repo isolates those parts so they can be read, tested and reused.

## Architecture

```
                ┌─────────────┐
   question ───▶│   Agent     │  (plan → tool → observe → answer)
                └──────┬──────┘
             ┌─────────┴─────────┐
             ▼                   ▼
      ┌────────────┐      ┌──────────────┐
      │ RAG        │      │ SQL tool     │
      │ embed→store│      │  guard()     │  ← blocks DML/DDL, enforces LIMIT
      │ →retrieve  │      └──────┬───────┘
      └─────┬──────┘             ▼
            │               ┌──────────┐
            ▼               │ SQLite   │
      ┌──────────┐          └──────────┘
      │ corpus   │
      └──────────┘

      MCP server (stdio, JSON-RPC 2.0)
      tools: search_docs · run_sql · list_tables
```

## Quickstart

No third-party runtime dependencies (Python 3.10+).

```bash
# run the offline evaluation harness
python eval/run_eval.py

# ask the agent a question
PYTHONPATH=src python -m rag_mcp_demo.cli "how many customers are there"
PYTHONPATH=src python -m rag_mcp_demo.cli "how does the RAG pipeline retrieve chunks"

# run the MCP server over stdio
PYTHONPATH=src python -m rag_mcp_demo.mcp_server
```

Install as a package (optional):

```bash
pip install -e ".[dev]"
rag-agent "total paid revenue"
pytest
```

## Example: MCP over stdio

```bash
echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | PYTHONPATH=src python -m rag_mcp_demo.mcp_server
```

## Evaluation

`eval/run_eval.py` reports two signals and exits non-zero if either drops below 0.8:

| Metric | Meaning |
|---|---|
| **retrieval recall@3** | fraction of golden questions whose expected document is in the top-3 |
| **sql guard accuracy** | fraction of safe/unsafe SQL cases classified correctly |

## Design decisions

1. **Vector store behind an interface** — in-memory for tests, Qdrant/pgvector in production.
2. **Safety before execution** — the SQL gate rejects non-`SELECT` statements and enforces a row limit; in production this is built on a real parser (e.g. `sqlglot`).
3. **Tools are read-only by default** — the MCP server exposes only read operations.
4. **Evals are first-class** — a golden set and a harness that fails CI on regression.
5. **LLM-agnostic agent** — the router is heuristic so the control flow is testable without a model; swap in an LLM to replace `Agent.route()`.

## Project layout

```
src/rag_mcp_demo/
  embeddings.py     # TF-IDF embedder + cosine (swap for a real model)
  vector_store.py   # in-memory vector store
  rag.py            # corpus loading + retrieval pipeline
  sql_guard.py      # pre-execution SQL safety gate
  db.py             # sample SQLite database
  agent.py          # minimal agent loop
  mcp_server.py     # MCP server (stdio, JSON-RPC 2.0)
  cli.py            # `rag-agent` CLI
eval/               # golden set + evaluation harness
tests/              # pytest suite
data/corpus.jsonl   # sample documents
```

## License

MIT — see [LICENSE](LICENSE).