# MCP-Powered Agentic RAG System

A local, modular Retrieval-Augmented Generation (RAG) system using the Model Context Protocol (MCP) to connect an LLM to external tools like vector databases and document loaders.


<img src="/images/mcp-agentic-rag.gif" alt="description" style="width:100%; height:auto;" />

## Overview

This project implements an agentic RAG system that:
- **Retrieves** relevant documents from a local vector database (ChromaDB)
- **Augments** prompts with retrieved context
- **Generates** informed responses using a local LLM (Ollama)
- **Exposes** functionality via REST API with FastAPI

## Tech Stack

| Component | Tool/Library | Details |
|-----------|---|---|
| **Language Model** | Ollama | Local LLM inference (mistral, llama3, etc.) |
| **Agent Framework** | mcp + FastAPI | API server with tool registration |
| **RAG Pipeline** | LangChain + Custom | Context retrieval and prompt engineering |
| **Vector Store** | ChromaDB | Local, persistent vector database |
| **Embeddings** | SentenceTransformers | all-MiniLM-L6-v2 model |
| **File Handling** | pypdf, python-docx | PDF and document loading |
| **Frontend (Optional)** | Streamlit | Interactive web UI |
| **Environment** | Python 3.10+ | virtualenv or Conda |

## Project Structure

```
agentic-rag-mcp/
├── main.py                    # FastAPI MCP server
├── rag_agent.py              # Agent query logic and RAG orchestration
├── mcp_config.yaml           # Configuration file
├── requirements.txt          # Python dependencies
├── vector_store/             # Persisted ChromaDB vector store
├── data/
│   └── sample_docs/          # Sample documents for ingestion
└── tools/
    └── chromadb_tool.py      # Vector search tool implementation
```

## Installation & Setup

### 1. Clone and Create Virtual Environment

```bash
cd agentic-rag-mcp
python -m venv .venv

# On Windows
.venv\Scripts\activate

# On macOS/Linux
source .venv/bin/activate
```

### 2. Install Dependencies

```bash
pip install -U pip
pip install -r requirements.txt
```

### 3. Set Up Ollama

Download and install [Ollama](https://ollama.ai) from the official website.

Start the Ollama server:

```bash
# On the system terminal (not in virtual environment)
ollama serve
```

In another terminal, pull a model:

```bash
ollama pull mistral    # Recommended for RAG
# or
ollama pull llama3
```

Verify the server is running:

```bash
curl http://localhost:11434/api/tags
```

## Running the System

### Option 1: Chat Interface (Interactive)

Run the interactive chat loop:

```bash
python rag_agent.py
```

This will:
1. Load sample documents into the vector store
2. Start an interactive chat where you can ask questions
3. The agent will retrieve relevant documents and generate answers

Example interaction:
```
You: What is MCP?
Agent: The Model Context Protocol (MCP) enables modular tool use for AI agents by providing a standardized way to connect language models to external services...

[Used 2 retrieved documents as context]
```

### Option 2: API Server

Start the FastAPI MCP server:

```bash
python main.py
```

The server will be available at: `http://localhost:8000`

### API Endpoints

#### Health Check
```bash
GET /health
```

#### Query Agent
```bash
POST /query
Content-Type: application/json

{
  "query": "What is artificial intelligence?",
  "use_context": true,
  "n_results": 3
}
```

#### Search Documents
```bash
POST /search
Content-Type: application/json

{
  "query": "MCP protocol",
  "n_results": 5
}
```

#### Add Documents
```bash
POST /documents
Content-Type: application/json

{
  "documents": [
    "Document text 1",
    "Document text 2"
  ],
  "ids": ["doc1", "doc2"],
  "metadata": [
    {"source": "file1.txt"},
    {"source": "file2.txt"}
  ]
}
```

#### Get Statistics
```bash
GET /stats
```

### Python Usage Examples

```python
from rag_agent import RAGAgent

# Initialize agent
agent = RAGAgent(
    ollama_url="http://localhost:11434",
    model="mistral"
)

# Get a response
result = agent.get_response("What is RAG?")
print(result["response"])
print(f"Retrieved {len(result['retrieved_documents'])} documents")
```

## Configuration

Edit `mcp_config.yaml` to customize:

- **LLM Settings**: Model, temperature, max tokens
- **Vector Store**: Embedding model, collection name
- **RAG**: Number of retrieved documents, similarity metric
- **Server**: Host, port, log level
- **Security**: API rate limits, authentication

## Adding Custom Documents

### Programmatically

```python
from tools.chromadb_tool import ChromaTool

tool = ChromaTool()
documents = [
    "Your document text 1",
    "Your document text 2"
]
tool.add_documents(documents, ids=["id1", "id2"])
```

### Via API

```bash
curl -X POST http://localhost:8000/documents \
  -H "Content-Type: application/json" \
  -d '{
    "documents": ["Document 1", "Document 2"],
    "ids": ["doc1", "doc2"]
  }'
```

## Optional Streamlit Frontend

Create `streamlit_app.py`:

```python
import streamlit as st
import requests

st.set_page_config(page_title="RAG Agent", layout="wide")
st.title("MCP-Powered Agentic RAG")

query = st.text_input("Ask a question:")

if query:
    response = requests.post(
        "http://localhost:8000/query",
        json={"query": query}
    )
    result = response.json()
    
    st.subheader("Response")
    st.write(result["response"])
    
    st.subheader("Retrieved Context")
    for i, doc in enumerate(result["retrieved_documents"], 1):
        st.write(f"**Doc {i}**: {doc[:200]}...")
```

Run Streamlit:
```bash
streamlit run streamlit_app.py
```

## Extensions & Future Work

- ✅ Basic RAG with ChromaDB
- ⬜ Web search tool integration
- ⬜ PDF document ingestion UI
- ⬜ Agent memory (conversation history)
- ⬜ Multi-modal support (images, tables)
- ⬜ Fine-tuning on domain-specific data
- ⬜ Structured output (JSON schemas)
- ⬜ Real-time streaming responses

## Troubleshooting

### "Connection refused" for Ollama
- Make sure Ollama server is running: `ollama serve`
- Check it's accessible: `curl http://localhost:11434/api/tags`

### ChromaDB embedding errors
- Ensure sentence-transformers is installed: `pip install sentence-transformers`
- First run downloads embeddings model (~30MB)

### Vector store not persisting
- Check `./vector_store/` directory exists and is writable
- Verify `persist_dir` in configuration matches actual path

## License

MIT License - See LICENSE file for details

## Contributing

Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Commit changes
4. Push and open a pull request

## References

- [Model Context Protocol](https://modelcontextprotocol.io/)
- [Ollama Documentation](https://ollama.ai)
- [ChromaDB Documentation](https://docs.trychroma.com/)
- [LangChain RAG Guide](https://python.langchain.com/docs/use_cases/question_answering/)
- [FastAPI Documentation](https://fastapi.tiangolo.com/)