rag-mcp
by MasihMoafi
README.md
---
name: rag-mcp
type: local semantic-search MCP server for coding agents and document workflows
---
<div align="center">
<img src="assets/retrieval-hero.svg" alt="rag-mcp retrieval reliability: two best of six recorded corpora" width="720">
<br>
[](#quick-start)
[](#what-sets-this-apart)
[](#architecture)
[](LICENSE)
**A coding agent should not have to choose between opening files one at a time and dumping an entire repository into context.**
[Install](#quick-start) • [Evals](#benchmark--retrieval-evaluations) • [Architecture](#architecture) • [How it works](#how-it-works) • [Deep-Dive Post](https://masihmoafi.com/projects/rag-mcp)
</div>
## Repository Layout
```text
rag-mcp/
├── rag/ # Core Hybrid RAG Pipeline & Storage Backends
│ ├── core.py # 2-Stage pipeline: Chunking, Ollama Embed, RRF Hybrid & Reranking
│ ├── lancedb_backend.py # LanceDB vector table + Tantivy FTS index integration
│ ├── qdrant_backend.py # Alternative Qdrant vector store backend
│ ├── fetch.py # Document ingestion, PDF, OCR & multiformat parser
│ └── notebook_chunker.py # Jupyter Notebook cell & code-block specialized chunker
│
├── tests/ # Pytest Automated Test Suite (18 passed)
│ ├── unit/ # Chunking logic, breadcrumb generation & GPU guardrail unit tests
│ ├── integration/ # LanceDB CRUD, MCP tool contracts, OCR & file scope tests
│ └── benchmarks/ # GPU empirical throughput sweeps & OOM ladder benchmarks
│
├── evals/ # Evaluation Datasets & Scaling Experiments
│ ├── datasets/ # 8 Clean ground-truth evaluation corpora & CSV query files
│ │ ├── attention/ # Attention Is All You Need (AI / scientific paper)
│ │ ├── brain-and-behavior/ # Neuroscience textbook (dense structured academic text)
│ │ ├── fire-and-blood/ # George R.R. Martin fantasy novel (narrative fiction)
│ │ ├── napoleon/ # Historical biography & philosophy
│ │ ├── mixed-codebase/ # Multi-language codebase (Python, Rust, Notebooks)
│ │ ├── elpis-memories-crate/ # Rust systems codebase
│ │ └── notebook/ # JSON / Jupyter scientific notebooks
│ └── experiments/ # Multi-Domain Scaling Benchmarks
│ └── experiment-unified-scaling/ # Scaling experiment logs, candidate outputs & LLM judge evals
│
├── server.py # FastMCP server entry point exposing tools to coding agents
└── pyproject.toml # Project metadata, dependencies (LanceDB, PyTorch, PyMuPDF)
```
## Benchmark & Retrieval Evaluations
Evaluated against the **[open-rag-eval](https://github.com/vectara/open-rag-eval)** taxonomy ($top\_k=5$, isolated local retrieval with no reranker, local `qwen3-embedding:8b` via Ollama + BM25 hybrid search). Full methodology in [`evals/README.md`](evals/README.md):
| Corpus / Domain | Total Queries | Strict Relevance (Score 3 / Exact) | Lenient Relevance (Score $\ge$ 2 / Full+Partial) | Miss Rate (Score $\le$ 1 / Miss) |
| :--- | :--- | :--- | :--- | :--- |
| **Attention Paper** (Scientific / AI) | 30 | **86.7%** (26/30) | **96.7%** (29/30) | 3.3% (1/30) |
| **Brain & Behavior** (Neuroscience) | 30 | **76.7%** (23/30) | **93.3%** (28/30) | 6.7% (2/30) |
| **Napoleon V2** (1000-char hybrid) | 30 | **66.7%** (20/30) | **90.0%** (27/30) | 10.0% (3/30) |
| **Napoleon V1** (300-char chunks) | 30 | **53.3%** (16/30) | **83.3%** (25/30) | 16.7% (5/30) |
| **Fire & Blood** (Narrative Fiction) | 30 | **46.7%** (14/30) | **80.0%** (24/30) | 20.0% (6/30) |
| **Mixed Codebase** (Py/Rust/IPYNB) | 33 | **90.9%** (30/33) | **100.0%** (33/33) | 0.0% (0/33) |
| **Elpis Memories Crate** (Rust) | 15 | **73.3%** (11/15) | **100.0%** (15/15) | 0.0% (0/15) |
| **rag-mcp Codebase** (Python Server) | 15 | **80.0%** (12/15) | **93.3%** (14/15) | 6.7% (1/15) |
| **Notebook Corpus** (JSON/Code) | 10 | **100.0%** (10/10) | **100.0%** (10/10) | 0.0% (0/10) |
| **Overall Baseline** | **203** | **74.9%** (152/203) | **92.1%** (187/203) | 7.9% (16/203) |
### Multi-Domain Scaling Experiment
Detailed evaluation log: [`evals/experiments/experiment-unified-scaling/experiment.md`](evals/experiments/experiment-unified-scaling/experiment.md) and [`Napoleon Experiment Log`](evals/experiments/experiment-unified-scaling/data/napoleon/experiment_log.md).
Evaluated across **5 merged heterogeneous domains (6,314 chunks in a single index)** comparing isolated baselines against unified scaling on NVIDIA GPU with `qwen3-embedding:0.6b` + `cross-encoder/ms-marco-MiniLM-L-6-v2` reranker:
| Corpus / Domain | Queries | Isolated Baseline Hit@5 | Unified Scaled Hit@5 | Isolated Baseline MRR | Unified Scaled MRR | Domain Purity in Unified |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **Attention Paper** (Scientific / AI) | 30 | **90.0%** (27/30) | **90.0%** (27/30) | 0.803 | 0.803 | 96.7% |
| **Brain & Behavior** (Neuroscience) | 30 | **93.3%** (28/30) | **93.3%** (28/30) | 0.831 | 0.831 | 94.7% |
| **Napoleon Biography** (History / Phil) | 30 | **66.7%** (20/30) | **66.7%** (20/30) | 0.558 | 0.548 | 100.0% |
| **Fire & Blood** (Narrative Fiction) | 30 | **46.7%** (14/30) | **46.7%** (14/30) | 0.372 | 0.372 | 100.0% |
| **Mixed Codebase** (Py/Rust/IPYNB) | 33 | **87.9%** (29/33) | **87.9%** (29/33) | 0.812 | 0.812 | 100.0% |
| **Overall Experiment 1 Total** | **153** | **77.1%** (118/153) | **77.1%** (118/153) | **0.675** | **0.673** | **98.4%** |
*Key finding: Merging 5 domains into one 6,314-chunk database produces **0.0% retrieval degradation** with **98.4% domain isolation purity** and **368ms average latency**.*
> [!TIP]
> Read the complete architectural walkthrough and benchmark breakdown on the [official blog post](https://masihmoafi.com/projects/rag-mcp).
<br>
## Quick start
Prerequisites: Python 3.10+ and [`uv`](https://docs.astral.sh/uv/getting-started/installation/).
```bash
git clone https://github.com/MasihMoafi/rag-mcp
cd rag-mcp
python scripts/bootstrap.py
```
That single command installs dependencies and runs the test suite.
Manual equivalent:
```bash
uv sync --group dev
.venv/bin/python -m pytest tests/ -v
```
Then register the server with an MCP client.
### Claude Code
```bash
claude mcp add rag -s user -- /absolute/path/to/rag-mcp/.venv/bin/python /absolute/path/to/rag-mcp/server.py
```
### Elpis
**Option A: Global Config (`~/.elpis/config.toml`)**
```toml
[mcp_servers.rag]
command = "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python"
args = ["/home/masih/Desktop/p/rag-mcp-lancedb/server.py"]
[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/home/masih/Desktop/p"
RAG_MCP_BACKEND = "lancedb"
```
**Option B: Standard MCP JSON (`.mcp.json` or `~/.elpis/mcp.json`)**
```json
{
"mcpServers": {
"rag": {
"type": "stdio",
"command": "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python",
"args": [
"/home/masih/Desktop/p/rag-mcp-lancedb/server.py"
],
"env": {
"RAG_MCP_WORKSPACE_ROOT": "/home/masih/Desktop/p",
"RAG_MCP_BACKEND": "lancedb"
}
}
}
}
```
### Codex
Add to `~/.codex/config.toml`:
```toml
[mcp_servers.rag]
command = "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python"
args = ["/home/masih/Desktop/p/rag-mcp-lancedb/server.py"]
[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/home/masih/Desktop/p"
RAG_MCP_BACKEND = "lancedb"
```
Expected result: the client discovers `query_knowledge_base`, and a query returns ranked passages with source paths from the requested scope.
## What is rag-mcp
`rag-mcp` is a local hybrid-search MCP server: point it at a file or folder, ask a question in plain language, and get back the passages that actually answer it — with the exact file and location, not a guess.
Coding agents normally search a codebase by opening files one at a time or dumping an entire repository into the conversation. Both waste time and context. `rag-mcp` replaces that with one tool call: search by meaning, get ranked results with sources, keep going. Embeddings, vector search, and reranking all run locally; the server exposes one read-only MCP tool to compatible clients.
No third-party logo or benchmark — the image above is the actual [retrieval-accuracy evidence](#retrieval-accuracy) this repo ships, not decoration.
## Retrieval accuracy
Recorded, reproducible retrieval runs live under [`evals/`](evals/) — not part of CI, kept
as evidence. Every point in the chart is a question-level grade from a recorded
`query_knowledge_base` run, graded against a known answer — not simulated.
The top result is a single real directory containing
a 29-file Rust crate, 6 Python scripts, and a Jupyter notebook, searched with `doc_path`
pointed at the whole directory — the server has to find the right file among three
languages, not just the right passage in one document. The other five rows are
single-document or project-scoped runs: a notebook, two books, a paper, a novel, and
the rag-mcp codebase itself.
The chart reports exact full answers separately from answers that were full or partial.
The codebase and structured-document rows are easier to retrieve exactly; the long
narrative rows contain more interpretive answers spread across passages, which makes
chunk-based retrieval less reliable. This is a description of these recorded runs, not
a claim about every corpus or a benchmark against other retrieval tools.
[Per-question results for every corpus](evals/) — including where each miss actually failed
(vague topical overlap, a truncated chunk, or the fact genuinely absent from top-k).
An earlier, now-superseded Napoleon run with reranking manually disabled scored 93.3% —
that number describes raw BM25+vector retrieval in isolation, not this server as it actually
runs, and is kept in [`evals/napoleon/experiment_log.md`](evals/napoleon/experiment_log.md)
only for its chunk-size finding (1000 characters beat 300). The earlier separate Rust-only
and notebook-only runs are likewise superseded by the combined directory test above and kept
under `evals/elpis-memories-crate/` and `evals/notebook/` only as raw evidence.
This is six recorded corpora at one point in time — not a benchmark against other
retrieval tools.
## The problem
Coding agents commonly retrieve context by either opening files one by one or loading a large portion of the repository. The first can miss relevant files; the second consumes context with material the current task may not need.
`rag-mcp` moves retrieval into one local tool call so the agent can search by meaning without making the entire tree part of every prompt.
## Architecture
<img src="assets/architecture.svg" alt="rag-mcp technical retrieval pipeline architecture" width="100%">
The retrieval pipeline operates across two decoupled stages:
1. **Scope Ingestion & Storage**: Target files matching the allowlist (`.rs`, `.py`, `.ts`, `.md`, `.ipynb`, `.pdf`) are processed through structure-aware AST chunking (`chunk_document`), preserving markdown heading hierarchies, code symbol scopes, and notebook cells. Chunks are simultaneously indexed into LanceDB's dense vector table (`all-MiniLM-L6-v2`) and embedded Tantivy inverted full-text search (FTS) index.
2. **Dual Retrieval & Precision Funnel**: Each `query_knowledge_base` invocation executes parallel dense vector cosine search and lexical Tantivy BM25 queries ($top\_k=20$). Candidates are unified via Reciprocal Rank Fusion ($\text{RRF}, k=60$), then passed through a neural Cross-Encoder (`cross-encoder/ms-marco-MiniLM-L-6-v2`) for full-attention relevance re-scoring before delivering ranked source citations.
## How it works
Repository structure:
```text
rag-mcp/
├── server.py # stdio JSON-RPC MCP host
├── rag/ # chunking, BM25, vector search, reranking
└── utils/proxy.py # local proxy-environment handling
```
Technical boundaries:
- one MCP tool: `query_knowledge_base(query, doc_path?)`;
- default embeddings: `all-MiniLM-L6-v2` (~80MB, fast — overridable, see Configuration);
- reranking runs by default: `cross-encoder/ms-marco-MiniLM-L-6-v2` (~80MB, fast — overridable, or disable it entirely);
- local embedded/on-disk Qdrant;
- `doc_path` can scope each call to a file or directory;
- per-path indexes are persisted under `rag/rag_db_v2/`;
- common large/build directories such as `.git`, `node_modules`, `.venv`, `dist`, `build`, and `target` are rejected;
- configurable depth/token limits fail explicitly instead of scanning an unbounded tree.
### Configuration
Every retrieval knob is an environment variable, not a source edit. Unset means the
default shown:
| Variable | Default | What it controls |
| --- | --- | --- |
| `RAG_MCP_EMBED_PROVIDER` | `sentencetransformer` | `sentencetransformer` (local), `ollama` (local server), or `openai_compatible` (remote — see below) |
| `RAG_MCP_EMBED_MODEL` | `all-MiniLM-L6-v2` | embedding model name, meaning depends on provider |
| `RAG_MCP_EMBED_API_KEY` | unset | API key, only used by `openai_compatible` |
| `RAG_MCP_EMBED_BASE_URL` | unset (official OpenAI endpoint) | override endpoint, only used by `openai_compatible` |
| `RAG_MCP_RERANKER_TYPE` | `cross-encoder` | `cross-encoder` or `disabled` |
| `RAG_MCP_RERANKER_MODEL` | `cross-encoder/ms-marco-MiniLM-L-6-v2` | reranker model, used when type is `cross-encoder` |
| `RAG_MCP_TOP_K` | `5` | candidates pulled from vector search before fusion |
| `RAG_MCP_RERANK_TOP_K` | `5` | results kept after reranking |
| `RAG_MCP_CHUNK_SIZE` | `700` | characters per chunk before overlap |
| `RAG_MCP_CHUNK_OVERLAP` | `100` | characters shared between adjacent chunks |
| `RAG_MCP_RRF_K` | `60` | Reciprocal Rank Fusion constant |
| `RAG_MCP_MAX_DEPTH` | `20` | directory-scan depth limit |
| `RAG_MCP_MAX_TOKENS` | `2000000` | directory-scan size limit |
The shipped defaults use the lightweight, fast pair (`all-MiniLM-L6-v2` + `ms-marco-MiniLM-L-6-v2`, ~80MB each) for instant zero-dependency local operation out of the box.
To run the higher-capacity pipeline evaluated in the benchmark suite above, switch to Ollama Qwen embeddings:
```toml
[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/absolute/path/to/your/project"
RAG_MCP_EMBED_PROVIDER = "ollama"
RAG_MCP_EMBED_MODEL = "qwen3-embedding:8b"
```
`openai_compatible` is the one non-local option: it leaves the machine. One client
implementation covers real OpenAI, Ollama's own `/v1` endpoint, and Qwen/DashScope's and
Gemini's OpenAI-compatible modes — install the optional `openai` package
(`uv sync --extra openai`), then set `RAG_MCP_EMBED_PROVIDER=openai_compatible`,
`RAG_MCP_EMBED_MODEL` to the provider's model name, `RAG_MCP_EMBED_API_KEY`, and
`RAG_MCP_EMBED_BASE_URL` if the provider isn't OpenAI itself. The client construction is
verified; a real embedding call against a paid provider is not — test it against your own
key before trusting it. A cross-encoder reranker type named `llm` also exists in the code
but its scoring is unimplemented scaffolding (every passage gets the same score) — do not
set it, it does nothing useful.
## Current state
### Implemented and verified
- MCP `initialize` → `tools/list` → `tools/call` protocol path.
- Read-only `query_knowledge_base` tool.
- Workspace-root and explicit `doc_path` scoping.
- Local hybrid retrieval and reranking.
- Guardrails for excluded directories and oversized scopes.
- Text extraction for PDF and Jupyter notebooks, plus local OCR for common image
formats and structured extraction for DOCX, PPTX, XLSX, and CSV files.
- End-to-end registration was exercised through a real MCP client during development.
### Implemented but not yet covered by the current tests
- The alternative Ollama embedding-provider path in `rag/core.py`.
- The `openai_compatible` embedding provider: client construction is verified, a real
call against a paid provider is not.
### Planned
Nothing is formally tracked yet. Extend it when a concrete retrieval failure or client requirement appears.
### Intentionally unsupported
- Hosted/remote vector databases.
- File types outside the extension allowlist in `server.py`.
- Write/mutation tools; this server is retrieval-only.
## What sets this apart
These are design choices, not novelty claims:
- **Local retrieval:** source files, embeddings, vector search, and reranking stay on the machine.
- **Small transport layer:** the MCP host uses direct stdio JSON-RPC rather than depending on an MCP SDK.
- **Per-call scope:** one server can search different files/directories instead of requiring one fixed knowledge base per project.
- **Evidence in the response:** returned chunks include source paths rather than only synthesized prose.
## Evals and test series
The test suite lives under `tests/`:
```bash
python scripts/bootstrap.py
```
They cover:
- read-only tool annotations;
- default workspace scoping;
- explicit `doc_path` scoping;
- rejection of excluded directories;
- rejection of depth-limit violations;
- bootstrap prerequisite checks;
- extraction from DOCX, PPTX, XLSX, and CSV files;
- local OCR extraction from a generated image fixture.
Protocol-level check, without another MCP client:
```bash
printf '%s\n%s\n%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
'{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"query_knowledge_base","arguments":{"query":"how does reciprocal rank fusion combine bm25 and vector results"}}}' \
| RAG_MCP_WORKSPACE_ROOT="$PWD" .venv/bin/python server.py
```
A successful self-query should return evidence pointing at the RRF implementation in `rag/core.py`.
What the tests prove: MCP transport/scoping/guardrail behavior covered by those cases.
What they do **not** prove: retrieval quality across arbitrary corpora, cross-client compatibility, or superiority to grep/code-search/RAG alternatives.
## Example
```text
query_knowledge_base(
"how does retry backoff work for failed jobs",
doc_path="codex-rs/memories"
)
```
The response is intended for the calling agent: ranked source passages it can use as task context rather than a standalone chat answer.
## Future development
Keep the surface small. Add capability only when real usage shows a retrieval, compatibility, or performance gap worth testing.
## License
MIT — see [LICENSE](LICENSE).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues