Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@AI Research Paper Assistant What is the attention mechanism in Transformers?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A Retrieval-Augmented Generation (RAG) system that answers questions about foundational AI/ML research papers, with an agentic reasoning layer, automated evaluation, and an MCP (Model Context Protocol) server so any MCP-compatible LLM client can use it as a tool.
Ask questions like:
"What is the attention mechanism in Transformers?"
"How does RAG combine retrieval with generation?"
"Compare how the Transformer paper and ReAct paper approach reasoning"
The system retrieves relevant passages from three indexed papers — Attention Is All You Need, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, and ReAct: Synergizing Reasoning and Acting in Language Models — and generates a grounded answer, citing the source paper and page.
For complex, multi-part questions, an agent layer breaks the question into sub-questions, retrieves context for each, and synthesizes a combined answer — rather than relying on a single retrieval pass.
Related MCP server: Research Citations MCP Server
┌─────────────┐
│ papers/ │ Raw PDFs (downloaded from arXiv)
└──────┬──────┘
│ ingest.py: load → chunk → embed
▼
┌─────────────┐
│ chroma_db/ │ Vector store (291 chunks, all-MiniLM-L6-v2 embeddings)
└──────┬──────┘
│
├─────────────────┬─────────────────────┐
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌────────────────────┐
│ rag_chain.py│ │ agent.py │ │ evaluation.py │
│ Simple LCEL │ │ LangGraph: │ │ RAGAS metrics: │
│ retrieve→ │ │ plan→ │ │ faithfulness, │
│ generate │ │ retrieve→ │ │ answer relevancy, │
│ │ │ synthesize │ │ context precision │
└──────┬──────┘ └──────┬───────┘ └─────────────────────┘
│ │
└───────┬────────┘
▼
┌─────────────────┐
│ mcp_server.py │ Exposes both as MCP tools
│ + app.py (CLI) │ for external LLM clients
└─────────────────┘Why this shape: all shared setup (embeddings, vector store connection, LLM client) lives in one place, src/config.py. Every other module imports from it instead of redefining it — so changing the embedding model or LLM provider means editing one file, not four.
Component | Choice | Why |
Orchestration | LangChain (LCEL) | Industry-standard way to compose retrieval + prompt + LLM into a pipeline |
Agent framework | LangGraph | Lets the system plan multi-step retrieval instead of one fixed pass — needed for comparison-style questions |
Vector store | Chroma | Local, free, simple to set up and inspect for a single-developer project |
Embeddings | HuggingFace | Runs locally on CPU, free, no API key or vendor lock-in — deliberate alternative to OpenAI embeddings |
LLM | Groq (Llama 3.3 70B) | Free tier, very fast inference, good enough quality for this scope |
Evaluation | RAGAS | Automated, LLM-judged scoring instead of just eyeballing answers |
Tool exposure | MCP (Model Context Protocol) | Turns the pipeline into a callable tool for any MCP client (e.g. Claude Desktop), not just a terminal script |
Measured with RAGAS on a 3-question test set against known ground-truth answers:
Metric | Score | What it measures |
Faithfulness | 0.87 | Does the answer stick to what's actually in the retrieved context, or hallucinate? |
Answer Relevancy | 0.93 | Does the answer actually address the question asked? |
Context Precision | 0.94 | Were the most relevant chunks ranked near the top of retrieval? |
ai-research-paper-assistant/
├── src/
│ ├── config.py # Single source of truth: embeddings, vectorstore, LLM setup
│ ├── ingest.py # PDF loading, chunking, embedding into Chroma
│ ├── rag_chain.py # Simple retrieve-then-generate LCEL chain
│ ├── agent.py # LangGraph agent: plan → retrieve → synthesize
│ └── evaluation.py # RAGAS evaluation harness
├── mcp_server.py # MCP tool wrapper around rag_chain and agent
├── app.py # CLI entry point (ingest / ask / agent / evaluate)
├── download_papers.py # One-time script to pull papers from arXiv
├── requirements.txt
└── eval_results.txt # Saved RAGAS output# 1. Clone and enter the project
git clone <your-repo-url>
cd ai-research-paper-assistant
# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# 3. Install dependencies
pip install -r requirements.txt
# 4. Add your Groq API key (free at console.groq.com)
echo "GROQ_API_KEY=your_key_here" > .env
# 5. Download the papers and build the vector store
python download_papers.py
python app.py ingestKnown dependency issue: ragas==0.3.9 ships with a broken import (langchain_community.chat_models.vertexai, a module path that's since moved). If you hit ModuleNotFoundError on that import when running evaluate, it's a known upstream bug, not a problem with this code — open an issue or check for a patched ragas release. A workaround is to wrap that one import line in the installed package in a try/except ImportError, since this project doesn't use VertexAI at all.
python app.py ask # Simple RAG: ask direct questions, one retrieval pass
python app.py agent # Agentic RAG: handles multi-part/comparison questions
python app.py evaluate # Re-run RAGAS scoring, saves to eval_results.txt
python mcp_server.py # Start as an MCP server for external clients (e.g. Claude Desktop)This server cannot be deployed
Query your own PDFs and documents from any MCP client. Every answer cites the page it came from.
Agent-native MCP server over the public saagarpatel.dev corpus. Read-only, stateless.
Agentic search over your Dewey document collections from any MCP-compatible client.
Academic research MCP server for paper search, citation checks, graphs, and deep research.
