# MCP Agentic RAG

A production-ready Retrieval-Augmented Generation (RAG) system exposed as an
[MCP](https://modelcontextprotocol.io) tool. The system searches a local vector
knowledge base first; if the query doesn't match well enough, it falls back to
live web search. Either way, an LLM generates a grounded, cited answer.

---

## Architecture

```
User query (via MCP client e.g. Cursor, Claude Desktop)
          │
          ▼
  ┌───────────────────┐
  │   MCP Server      │  server.py
  │  ask_ml_knowledge │
  └────────┬──────────┘
           │
           ▼
  ┌───────────────────┐
  │  Embed query      │  nomic-embed-text-v1.5
  └────────┬──────────┘
           │
           ▼
  ┌───────────────────┐
  │  Search Qdrant    │  cosine similarity
  └────────┬──────────┘
           │
     score >= 0.45?
      /           \
    YES            NO
     │              │
     ▼              ▼
  RAG context   Bright Data
  (local KB)    web search
     │              │
     └──────┬───────┘
            ▼
   ┌─────────────────┐
   │  LLM generates  │  OpenAI / local model
   │  grounded answer│
   └────────┬────────┘
            ▼
   Answer + numbered sources
```

---

## Project Structure

```
mcp-agentic-rag/
├── data/                        # Knowledge base documents
│   ├── machine_learning.txt
│   ├── deep_learning.txt
│   ├── neural_networks.txt
│   ├── cnn.txt
│   ├── rnn.txt
│   ├── transformers.txt
│   ├── optimization.txt
│   └── regularization.txt
├── eval/
│   └── eval_retrieval.py        # Recall@3 evaluation (30 questions)
├── rag_code.py                  # EmbedData, QdrantVDB, Retriever, AnswerGenerator
├── ingest.py                    # One-time ingestion script
├── server.py                    # MCP server entry point
├── docker-compose.yml           # Qdrant vector database
├── requirements.txt
├── .env.example                 # Copy to .env and fill in credentials
└── notebook.ipynb               # Interactive exploration notebook
```

---

## Setup

### 1. Prerequisites

- Python 3.11+
- [Docker Desktop](https://www.docker.com/products/docker-desktop/)

### 2. Install dependencies

```bash
pip install -r requirements.txt
```

### 3. Configure environment

```bash
cp .env.example .env
```

Edit `.env` and fill in at minimum:

| Variable | Required | Description |
|---|---|---|
| `OPENAI_API_KEY` | Yes | OpenAI API key for answer generation |
| `OPENAI_MODEL` | No | Model name (default: `gpt-4o-mini`) |
| `OPENAI_BASE_URL` | No | Custom endpoint for local LLMs (Ollama, LM Studio) |
| `QDRANT_URL` | No | Qdrant URL (default: `http://localhost:6333`) |
| `SCORE_THRESHOLD` | No | Similarity cutoff for web fallback (default: `0.45`) |
| `BRIGHT_DATA_USERNAME` | No | Bright Data SERP credentials (web fallback only) |
| `BRIGHT_DATA_PASSWORD` | No | Bright Data SERP credentials (web fallback only) |

### 4. Start Qdrant

```bash
docker compose up -d
```

Qdrant dashboard: http://localhost:6333/dashboard

### 5. Ingest the knowledge base

```bash
python ingest.py
```

This embeds all documents in `data/` and uploads them to Qdrant.
Re-run with `--reset` after updating documents:

```bash
python ingest.py --reset
```

### 6. Register the MCP server

Add the following to your MCP client configuration (e.g. Cursor `mcp.json`,
Claude Desktop `claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "mcp-agentic-rag": {
      "command": "python",
      "args": ["/absolute/path/to/server.py"]
    }
  }
}
```

The server is now available. Your AI assistant will automatically call
`ask_ml_knowledge_base` when relevant.

---

## Using a Local LLM (no OpenAI account needed)

Install [Ollama](https://ollama.com) and pull a model:

```bash
ollama pull llama3.2
```

Set in `.env`:

```
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=ollama
OPENAI_MODEL=llama3.2
```

---

## Evaluation

Run the built-in Recall@3 evaluation on 30 domain-specific questions
(requires Qdrant running with the collection ingested):

```bash
python eval/eval_retrieval.py
```

Save full results to JSON:

```bash
python eval/eval_retrieval.py --output eval/results.json
```

Example output:

```
============================================================
  Results Summary
============================================================
  Total questions : 30
  Correct (hit)   : 26
  Missed          : 4
  Recall@3        : 26/30 = 86.7%
============================================================
```

---

## Adding More Documents

1. Create a `.txt` file in `data/` with this header format:

```
TITLE: Your Document Title
SOURCE: your_file.txt
URL: https://optional-reference-url.com

---

First chunk of text goes here.

---

Second chunk of text goes here.
```

2. Re-ingest:

```bash
python ingest.py --reset
```

---

## License

MIT