AI Research Paper Assistant
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@AI Research Paper AssistantWhat is the attention mechanism in Transformers?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
AI Research Paper Assistant
A Retrieval-Augmented Generation (RAG) system that answers questions about foundational AI/ML research papers, with an agentic reasoning layer, automated evaluation, and an MCP (Model Context Protocol) server so any MCP-compatible LLM client can use it as a tool.
What it does
Ask questions like:
"What is the attention mechanism in Transformers?"
"How does RAG combine retrieval with generation?"
"Compare how the Transformer paper and ReAct paper approach reasoning"
The system retrieves relevant passages from three indexed papers — Attention Is All You Need, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, and ReAct: Synergizing Reasoning and Acting in Language Models — and generates a grounded answer, citing the source paper and page.
For complex, multi-part questions, an agent layer breaks the question into sub-questions, retrieves context for each, and synthesizes a combined answer — rather than relying on a single retrieval pass.
Related MCP server: paper-fetch-mcp
Architecture
┌─────────────┐
│ papers/ │ Raw PDFs (downloaded from arXiv)
└──────┬──────┘
│ ingest.py: load → chunk → embed
▼
┌─────────────┐
│ chroma_db/ │ Vector store (291 chunks, all-MiniLM-L6-v2 embeddings)
└──────┬──────┘
│
├─────────────────┬─────────────────────┐
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌────────────────────┐
│ rag_chain.py│ │ agent.py │ │ evaluation.py │
│ Simple LCEL │ │ LangGraph: │ │ RAGAS metrics: │
│ retrieve→ │ │ plan→ │ │ faithfulness, │
│ generate │ │ retrieve→ │ │ answer relevancy, │
│ │ │ synthesize │ │ context precision │
└──────┬──────┘ └──────┬───────┘ └─────────────────────┘
│ │
└───────┬────────┘
▼
┌─────────────────┐
│ mcp_server.py │ Exposes both as MCP tools
│ + app.py (CLI) │ for external LLM clients
└─────────────────┘Why this shape: all shared setup (embeddings, vector store connection, LLM client) lives in one place, src/config.py. Every other module imports from it instead of redefining it — so changing the embedding model or LLM provider means editing one file, not four.
Tech stack and why each piece was chosen
Component | Choice | Why |
Orchestration | LangChain (LCEL) | Industry-standard way to compose retrieval + prompt + LLM into a pipeline |
Agent framework | LangGraph | Lets the system plan multi-step retrieval instead of one fixed pass — needed for comparison-style questions |
Vector store | Chroma | Local, free, simple to set up and inspect for a single-developer project |
Embeddings | HuggingFace | Runs locally on CPU, free, no API key or vendor lock-in — deliberate alternative to OpenAI embeddings |
LLM | Groq (Llama 3.3 70B) | Free tier, very fast inference, good enough quality for this scope |
Evaluation | RAGAS | Automated, LLM-judged scoring instead of just eyeballing answers |
Tool exposure | MCP (Model Context Protocol) | Turns the pipeline into a callable tool for any MCP client (e.g. Claude Desktop), not just a terminal script |
Evaluation results
Measured with RAGAS on a 3-question test set against known ground-truth answers:
Metric | Score | What it measures |
Faithfulness | 0.87 | Does the answer stick to what's actually in the retrieved context, or hallucinate? |
Answer Relevancy | 0.93 | Does the answer actually address the question asked? |
Context Precision | 0.94 | Were the most relevant chunks ranked near the top of retrieval? |
Project structure
ai-research-paper-assistant/
├── src/
│ ├── config.py # Single source of truth: embeddings, vectorstore, LLM setup
│ ├── ingest.py # PDF loading, chunking, embedding into Chroma
│ ├── rag_chain.py # Simple retrieve-then-generate LCEL chain
│ ├── agent.py # LangGraph agent: plan → retrieve → synthesize
│ └── evaluation.py # RAGAS evaluation harness
├── mcp_server.py # MCP tool wrapper around rag_chain and agent
├── app.py # CLI entry point (ingest / ask / agent / evaluate)
├── download_papers.py # One-time script to pull papers from arXiv
├── requirements.txt
└── eval_results.txt # Saved RAGAS outputSetup
# 1. Clone and enter the project
git clone <your-repo-url>
cd ai-research-paper-assistant
# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# 3. Install dependencies
pip install -r requirements.txt
# 4. Add your Groq API key (free at console.groq.com)
echo "GROQ_API_KEY=your_key_here" > .env
# 5. Download the papers and build the vector store
python download_papers.py
python app.py ingestKnown dependency issue: ragas==0.3.9 ships with a broken import (langchain_community.chat_models.vertexai, a module path that's since moved). If you hit ModuleNotFoundError on that import when running evaluate, it's a known upstream bug, not a problem with this code — open an issue or check for a patched ragas release. A workaround is to wrap that one import line in the installed package in a try/except ImportError, since this project doesn't use VertexAI at all.
Usage
python app.py ask # Simple RAG: ask direct questions, one retrieval pass
python app.py agent # Agentic RAG: handles multi-part/comparison questions
python app.py evaluate # Re-run RAGAS scoring, saves to eval_results.txt
python mcp_server.py # Start as an MCP server for external clients (e.g. Claude Desktop)This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/guntaasaneja/ai-research-paper-assistant'
If you have feedback or need assistance with the MCP directory API, please join our Discord server