Skip to main content
Glama
guntaasaneja

AI Research Paper Assistant

by guntaasaneja

AI Research Paper Assistant

A Retrieval-Augmented Generation (RAG) system that answers questions about foundational AI/ML research papers, with an agentic reasoning layer, automated evaluation, and an MCP (Model Context Protocol) server so any MCP-compatible LLM client can use it as a tool.


What it does

Ask questions like:

  • "What is the attention mechanism in Transformers?"

  • "How does RAG combine retrieval with generation?"

  • "Compare how the Transformer paper and ReAct paper approach reasoning"

The system retrieves relevant passages from three indexed papers — Attention Is All You Need, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, and ReAct: Synergizing Reasoning and Acting in Language Models — and generates a grounded answer, citing the source paper and page.

For complex, multi-part questions, an agent layer breaks the question into sub-questions, retrieves context for each, and synthesizes a combined answer — rather than relying on a single retrieval pass.


Related MCP server: paper-fetch-mcp

Architecture

┌─────────────┐
│   papers/   │  Raw PDFs (downloaded from arXiv)
└──────┬──────┘
       │  ingest.py: load → chunk → embed
       ▼
┌─────────────┐
│  chroma_db/ │  Vector store (291 chunks, all-MiniLM-L6-v2 embeddings)
└──────┬──────┘
       │
       ├─────────────────┬─────────────────────┐
       ▼                 ▼                      ▼
┌─────────────┐  ┌──────────────┐   ┌────────────────────┐
│ rag_chain.py│  │  agent.py    │   │  evaluation.py      │
│ Simple LCEL │  │  LangGraph:  │   │  RAGAS metrics:      │
│ retrieve→   │  │  plan→       │   │  faithfulness,        │
│ generate    │  │  retrieve→   │   │  answer relevancy,    │
│             │  │  synthesize  │   │  context precision    │
└──────┬──────┘  └──────┬───────┘   └─────────────────────┘
       │                │
       └───────┬────────┘
               ▼
      ┌─────────────────┐
      │  mcp_server.py   │  Exposes both as MCP tools
      │  + app.py (CLI)  │  for external LLM clients
      └─────────────────┘

Why this shape: all shared setup (embeddings, vector store connection, LLM client) lives in one place, src/config.py. Every other module imports from it instead of redefining it — so changing the embedding model or LLM provider means editing one file, not four.


Tech stack and why each piece was chosen

Component

Choice

Why

Orchestration

LangChain (LCEL)

Industry-standard way to compose retrieval + prompt + LLM into a pipeline

Agent framework

LangGraph

Lets the system plan multi-step retrieval instead of one fixed pass — needed for comparison-style questions

Vector store

Chroma

Local, free, simple to set up and inspect for a single-developer project

Embeddings

HuggingFace all-MiniLM-L6-v2

Runs locally on CPU, free, no API key or vendor lock-in — deliberate alternative to OpenAI embeddings

LLM

Groq (Llama 3.3 70B)

Free tier, very fast inference, good enough quality for this scope

Evaluation

RAGAS

Automated, LLM-judged scoring instead of just eyeballing answers

Tool exposure

MCP (Model Context Protocol)

Turns the pipeline into a callable tool for any MCP client (e.g. Claude Desktop), not just a terminal script


Evaluation results

Measured with RAGAS on a 3-question test set against known ground-truth answers:

Metric

Score

What it measures

Faithfulness

0.87

Does the answer stick to what's actually in the retrieved context, or hallucinate?

Answer Relevancy

0.93

Does the answer actually address the question asked?

Context Precision

0.94

Were the most relevant chunks ranked near the top of retrieval?


Project structure

ai-research-paper-assistant/
├── src/
│   ├── config.py       # Single source of truth: embeddings, vectorstore, LLM setup
│   ├── ingest.py        # PDF loading, chunking, embedding into Chroma
│   ├── rag_chain.py     # Simple retrieve-then-generate LCEL chain
│   ├── agent.py          # LangGraph agent: plan → retrieve → synthesize
│   └── evaluation.py    # RAGAS evaluation harness
├── mcp_server.py         # MCP tool wrapper around rag_chain and agent
├── app.py                 # CLI entry point (ingest / ask / agent / evaluate)
├── download_papers.py     # One-time script to pull papers from arXiv
├── requirements.txt
└── eval_results.txt        # Saved RAGAS output

Setup

# 1. Clone and enter the project
git clone <your-repo-url>
cd ai-research-paper-assistant

# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Add your Groq API key (free at console.groq.com)
echo "GROQ_API_KEY=your_key_here" > .env

# 5. Download the papers and build the vector store
python download_papers.py
python app.py ingest

Known dependency issue: ragas==0.3.9 ships with a broken import (langchain_community.chat_models.vertexai, a module path that's since moved). If you hit ModuleNotFoundError on that import when running evaluate, it's a known upstream bug, not a problem with this code — open an issue or check for a patched ragas release. A workaround is to wrap that one import line in the installed package in a try/except ImportError, since this project doesn't use VertexAI at all.


Usage

python app.py ask         # Simple RAG: ask direct questions, one retrieval pass
python app.py agent       # Agentic RAG: handles multi-part/comparison questions
python app.py evaluate    # Re-run RAGAS scoring, saves to eval_results.txt
python mcp_server.py      # Start as an MCP server for external clients (e.g. Claude Desktop)
F
license - not found
-
quality - not tested
C
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/guntaasaneja/ai-research-paper-assistant'

If you have feedback or need assistance with the MCP directory API, please join our Discord server