# AI Research Paper Assistant

A Retrieval-Augmented Generation (RAG) system that answers questions about foundational AI/ML research papers, with an agentic reasoning layer, automated evaluation, and an MCP (Model Context Protocol) server so any MCP-compatible LLM client can use it as a tool.

---

## What it does

Ask questions like:
- "What is the attention mechanism in Transformers?"
- "How does RAG combine retrieval with generation?"
- "Compare how the Transformer paper and ReAct paper approach reasoning"

The system retrieves relevant passages from three indexed papers — *Attention Is All You Need*, *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks*, and *ReAct: Synergizing Reasoning and Acting in Language Models* — and generates a grounded answer, citing the source paper and page.

For complex, multi-part questions, an agent layer breaks the question into sub-questions, retrieves context for each, and synthesizes a combined answer — rather than relying on a single retrieval pass.

---

## Architecture

```
┌─────────────┐
│   papers/   │  Raw PDFs (downloaded from arXiv)
└──────┬──────┘
       │  ingest.py: load → chunk → embed
       ▼
┌─────────────┐
│  chroma_db/ │  Vector store (291 chunks, all-MiniLM-L6-v2 embeddings)
└──────┬──────┘
       │
       ├─────────────────┬─────────────────────┐
       ▼                 ▼                      ▼
┌─────────────┐  ┌──────────────┐   ┌────────────────────┐
│ rag_chain.py│  │  agent.py    │   │  evaluation.py      │
│ Simple LCEL │  │  LangGraph:  │   │  RAGAS metrics:      │
│ retrieve→   │  │  plan→       │   │  faithfulness,        │
│ generate    │  │  retrieve→   │   │  answer relevancy,    │
│             │  │  synthesize  │   │  context precision    │
└──────┬──────┘  └──────┬───────┘   └─────────────────────┘
       │                │
       └───────┬────────┘
               ▼
      ┌─────────────────┐
      │  mcp_server.py   │  Exposes both as MCP tools
      │  + app.py (CLI)  │  for external LLM clients
      └─────────────────┘
```

**Why this shape:** all shared setup (embeddings, vector store connection, LLM client) lives in one place, `src/config.py`. Every other module imports from it instead of redefining it — so changing the embedding model or LLM provider means editing one file, not four.

---

## Tech stack and why each piece was chosen

| Component | Choice | Why |
|---|---|---|
| Orchestration | LangChain (LCEL) | Industry-standard way to compose retrieval + prompt + LLM into a pipeline |
| Agent framework | LangGraph | Lets the system plan multi-step retrieval instead of one fixed pass — needed for comparison-style questions |
| Vector store | Chroma | Local, free, simple to set up and inspect for a single-developer project |
| Embeddings | HuggingFace `all-MiniLM-L6-v2` | Runs locally on CPU, free, no API key or vendor lock-in — deliberate alternative to OpenAI embeddings |
| LLM | Groq (Llama 3.3 70B) | Free tier, very fast inference, good enough quality for this scope |
| Evaluation | RAGAS | Automated, LLM-judged scoring instead of just eyeballing answers |
| Tool exposure | MCP (Model Context Protocol) | Turns the pipeline into a callable tool for any MCP client (e.g. Claude Desktop), not just a terminal script |

---

## Evaluation results

Measured with RAGAS on a 3-question test set against known ground-truth answers:

| Metric | Score | What it measures |
|---|---|---|
| Faithfulness | 0.87 | Does the answer stick to what's actually in the retrieved context, or hallucinate? |
| Answer Relevancy | 0.93 | Does the answer actually address the question asked? |
| Context Precision | 0.94 | Were the most relevant chunks ranked near the top of retrieval? |

---

## Project structure

```
ai-research-paper-assistant/
├── src/
│   ├── config.py       # Single source of truth: embeddings, vectorstore, LLM setup
│   ├── ingest.py        # PDF loading, chunking, embedding into Chroma
│   ├── rag_chain.py     # Simple retrieve-then-generate LCEL chain
│   ├── agent.py          # LangGraph agent: plan → retrieve → synthesize
│   └── evaluation.py    # RAGAS evaluation harness
├── mcp_server.py         # MCP tool wrapper around rag_chain and agent
├── app.py                 # CLI entry point (ingest / ask / agent / evaluate)
├── download_papers.py     # One-time script to pull papers from arXiv
├── requirements.txt
└── eval_results.txt        # Saved RAGAS output
```

---

## Setup

```bash
# 1. Clone and enter the project
git clone <your-repo-url>
cd ai-research-paper-assistant

# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Add your Groq API key (free at console.groq.com)
echo "GROQ_API_KEY=your_key_here" > .env

# 5. Download the papers and build the vector store
python download_papers.py
python app.py ingest
```

**Known dependency issue:** `ragas==0.3.9` ships with a broken import (`langchain_community.chat_models.vertexai`, a module path that's since moved). If you hit `ModuleNotFoundError` on that import when running `evaluate`, it's a known upstream bug, not a problem with this code — open an issue or check for a patched `ragas` release. A workaround is to wrap that one import line in the installed package in a `try/except ImportError`, since this project doesn't use VertexAI at all.

---

## Usage

```bash
python app.py ask         # Simple RAG: ask direct questions, one retrieval pass
python app.py agent       # Agentic RAG: handles multi-part/comparison questions
python app.py evaluate    # Re-run RAGAS scoring, saves to eval_results.txt
python mcp_server.py      # Start as an MCP server for external clients (e.g. Claude Desktop)
```