# MCP FastAPI Multi-Agent RAG System

A production-grade AI assistant system that combines retrieval-augmented generation (RAG), multi-agent orchestration, and the Model Context Protocol (MCP) to answer queries across documents, calculations, and live data.

## What This System Does

This architecture deploys an agentic system that decides which tool to use for a given query — similar to how a researcher might choose between consulting a reference library (documents), running calculations, or checking live weather data. The system ingests PDF documents, embeds them into a vector database, and serves a ReAct agent via FastAPI that can retrieve document context, perform arithmetic, fetch weather, and persist conversation history to SQLite.

---

## Architecture Overview

```
User Query (FastAPI)
    ↓
ReAct Agent (LangGraph + Ollama qwen2.5)
    ├─→ [Tool 1] Calculator (numexpr)
    ├─→ [Tool 2] Weather (wttr.in API)
    └─→ [Tool 3] Hybrid Retrieval (ChromaDB + BM25)
              ├─ Dense Search (SentenceTransformer embeddings)
              └─ Sparse Search (BM25 keyword ranking)
                   ↓
    Response Persisted to SQLite
```

### Key Components

**Hybrid Retrieval** — Combines dense semantic search (ChromaDB embeddings) and sparse lexical search (BM25 token scoring) via reciprocal rank fusion (RRF). Think of it as asking both "What does this mean semantically?" and "What keywords appear here?" and trusting neither alone.

**MCP Tools** — The Model Context Protocol lets the LLM invoke tools declaratively. Each tool (Calculator, Weather, Hybrid_Retrival) is registered with explicit docstrings that guide the agent on *when* to use it.

**LangGraph ReAct Agent** — Implements the Reasoning + Acting loop: the model thinks through which tool fits, invokes it, observes the result, and iterates until it produces a final response.

---

## Prerequisites

- Python 3.10+
- Ollama installed locally with `qwen2.5:1.5b` model pulled
- SQLite (included with Python)

---

## Installation & Setup

### 1. Clone and Install Dependencies

```bash
git clone <repo-url>
cd MCP\ FastAPI\ System
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

### 2. Prepare Documents

Place your PDF files in the project root. The default is `Why_Language_Models_Hallucinate_Explainer.pdf`. The system will:
- Load and chunk the PDF
- Generate MD5 IDs for each chunk
- Embed chunks with `all-MiniLM-L6-v2` (SentenceTransformer)
- Store in ChromaDB (persists to `MCP_DB/` directory)

### 3. Update File Paths

Edit `MCP_Client.py` and `MCP_Server.py` to match your environment:
```python
# Example: Update the absolute path to MCP_Server.py
"args": ["/your/path/to/MCP_Server.py"],
```

---

## Running the System

### Option 1: FastAPI Server (Recommended for Production)

```bash
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

Access the interactive docs at `http://localhost:8000/docs`.

### Option 2: Standalone Client

```bash
python MCP_Client.py
```

This runs an async query directly without the HTTP wrapper.

### Option 3: Jupyter Notebook

Run `ALL_to_gether.ipynb` cell by cell for development and debugging.

---

## API Endpoints

### POST `/chat` — Ask a Question
```json
{
  "Question": "What causes hallucination in language models?",
  "limit": 50
}
```

**Response:**
```json
{
  "id": 1,
  "question": "What causes hallucination in language models?",
  "limit": 50,
  "answer": "<Agent's response>"
}
```

### GET `/ask` — Retrieve All Queries
Returns all stored questions and answers from the database.

### PUT `/ask` — Update a Query
```json
{
  "ID": 1,
  "Question": "Updated question",
  "limit": 75
}
```

### DELETE `/ask` — Delete a Query
```json
{
  "ID": 1
}
```

---

## How It Works: A Concrete Example

**User Query:** "What is 25 * 8?"

1. **FastAPI** receives the question and invokes `MCP_Client.main()`.
2. **MCP Client** connects to the MCP Server (subprocess over stdio).
3. **Server** exposes three tools; the **ReAct Agent** receives them.
4. **Agent** reads the query, sees numeric operators, and calls the `Calculator` tool with `"25 * 8"`.
5. **Calculator** uses `numexpr.evaluate()` to return `200`.
6. **Agent** formats the response: `"200"`.
7. **FastAPI** stores the Q&A in SQLite and returns JSON.

---

## Configuration

### Model Selection
Edit the model name in `MCP_Server.py` or `MCP_Client.py`:
```python
ollama = ChatOllama(model="qwen2.5:1.5b")  # Change to qwen2.5:7b for higher accuracy
```

### Retrieval Parameters
Adjust in `MCP_Server.py`:
- `chunk_size=1500` — Size of each document chunk
- `chunk_overlap=180` — Overlap between consecutive chunks
- `threshold=1.0` — Distance threshold for dense retrieval
- `k=10` — Number of BM25 results to consider

### RRF Fusion
Tune the `rrf_tokes` calculation (currently using rank+60 denominator) to weight dense vs. sparse results differently.

---

## Project Structure

```
MCP FastAPI System/
├── MCP_Server.py          # Exposes Calculator, Weather, Hybrid_Retrival tools
├── MCP_Client.py          # Connects to server, builds ReAct agent, runs queries
├── main.py                # FastAPI app entry point
├── Router.py              # API endpoints (POST /chat, GET /ask, etc.)
├── DataBase.py            # SQLAlchemy models and session management
├── ALL_to_gether.ipynb    # Development notebook (client + server in one)
├── requirements.txt       # Dependencies
├── SQL.db                 # SQLite database (auto-created)
├── MCP_DB/                # ChromaDB persistence (auto-created)
└── Why_Language_Models_Hallucinate_Explainer.pdf
```

---

## Key Design Decisions

1. **Ollama (Local LLM)** — Runs inference locally; no API costs or latency bottlenecks.
2. **Hybrid Retrieval** — Single-modality (dense-only) search misses keyword-heavy queries; hybrid fusion catches both semantic and lexical matches.
3. **Tool Docstrings** — Each tool has explicit, detailed instructions. The agent reads these to decide *when* to invoke each tool.
4. **SQLite Persistence** — Simple, file-based; suitable for small to medium deployments. Swap for PostgreSQL for horizontal scaling.
5. **MCP over Stdio** — Server runs as a subprocess; communication happens via stdin/stdout, ensuring isolation and easy debugging.

---

## Limitations & Future Work

- **Ollama Availability** — System assumes a running Ollama instance. Graceful fallback to remote endpoints (e.g., Groq, Anthropic) planned.
- **Single PDF** — Currently loads one PDF; extend to multiple documents via a document registry.
- **No Caching** — Repeated queries re-run full RAG pipeline; Redis cache layer recommended for production.
- **Tool Routing Logic** — Relies on LLM judgment; adversarial queries may misroute. Add explicit regex pre-checks for calculator vs. general queries.

---

## Troubleshooting

**"MCP adapter working" doesn't print** → MCP server subprocess failed. Check:
```bash
python /path/to/MCP_Server.py
```

**ChromaDB errors** → Delete `MCP_DB/` and rerun to rebuild embeddings.

**Ollama model not found** → Pull the model:
```bash
ollama pull qwen2.5:1.5b
```

**Tool not invoked** → Check system prompt in `MCP_Client.py`. Agent follows tool docstrings; ensure they are clear and unambiguous.

---

## Contributing

Fork, experiment, and refactor. The system is designed for extension: add new tools by decorating functions with `@mcp.tool()` in `MCP_Server.py`, then rebuild the agent.