rag-mcp

name: rag-mcp type: local semantic-search MCP server for coding agents and document workflows

A coding agent should not have to choose between opening files one at a time and dumping an entire repository into context.

Install • Evals • Architecture • How it works • Deep-Dive Post

Related MCP server: embecode

Repository Layout

rag-mcp/
├── rag/                                # Core Hybrid RAG Pipeline & Storage Backends
│   ├── core.py                         # 2-Stage pipeline: Chunking, Ollama Embed, RRF Hybrid & Reranking
│   ├── lancedb_backend.py              # LanceDB vector table + Tantivy FTS index integration
│   ├── qdrant_backend.py               # Alternative Qdrant vector store backend
│   ├── fetch.py                        # Document ingestion, PDF, OCR & multiformat parser
│   └── notebook_chunker.py             # Jupyter Notebook cell & code-block specialized chunker
│
├── tests/                              # Pytest Automated Test Suite (18 passed)
│   ├── unit/                           # Chunking logic, breadcrumb generation & GPU guardrail unit tests
│   ├── integration/                    # LanceDB CRUD, MCP tool contracts, OCR & file scope tests
│   └── benchmarks/                     # GPU empirical throughput sweeps & OOM ladder benchmarks
│
├── evals/                              # Evaluation Datasets & Scaling Experiments
│   ├── datasets/                       # 8 Clean ground-truth evaluation corpora & CSV query files
│   │   ├── attention/                  # Attention Is All You Need (AI / scientific paper)
│   │   ├── brain-and-behavior/         # Neuroscience textbook (dense structured academic text)
│   │   ├── fire-and-blood/             # George R.R. Martin fantasy novel (narrative fiction)
│   │   ├── napoleon/                   # Historical biography & philosophy
│   │   ├── mixed-codebase/             # Multi-language codebase (Python, Rust, Notebooks)
│   │   ├── elpis-memories-crate/       # Rust systems codebase
│   │   └── notebook/                   # JSON / Jupyter scientific notebooks
│   └── experiments/                    # Multi-Domain Scaling Benchmarks
│       └── experiment-unified-scaling/ # Scaling experiment logs, candidate outputs & LLM judge evals
│
├── server.py                           # FastMCP server entry point exposing tools to coding agents
└── pyproject.toml                      # Project metadata, dependencies (LanceDB, PyTorch, PyMuPDF)

Benchmark & Retrieval Evaluations

Evaluated against the open-rag-eval taxonomy ($top_k=5$, isolated local retrieval with no reranker, local qwen3-embedding:8b via Ollama + BM25 hybrid search). Full methodology in evals/README.md:

Multi-Domain Scaling Experiment

Detailed evaluation log: evals/experiments/experiment-unified-scaling/experiment.md and Napoleon Experiment Log.

Evaluated across 5 merged heterogeneous domains (6,314 chunks in a single index) comparing isolated baselines against unified scaling on NVIDIA GPU with qwen3-embedding:0.6b + cross-encoder/ms-marco-MiniLM-L-6-v2 reranker:

Key finding: Merging 5 domains into one 6,314-chunk database produces 0.0% retrieval degradation with 98.4% domain isolation purity and 368ms average latency.

Quick start

Prerequisites: Python 3.10+ and uv.

git clone https://github.com/MasihMoafi/rag-mcp
cd rag-mcp
python scripts/bootstrap.py

That single command installs dependencies and runs the test suite.

Manual equivalent:

uv sync --group dev
.venv/bin/python -m pytest tests/ -v

Then register the server with an MCP client.

Claude Code

claude mcp add rag -s user -- /absolute/path/to/rag-mcp/.venv/bin/python /absolute/path/to/rag-mcp/server.py

Elpis

Option A: Global Config (~/.elpis/config.toml)

[mcp_servers.rag]
command = "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python"
args = ["/home/masih/Desktop/p/rag-mcp-lancedb/server.py"]

[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/home/masih/Desktop/p"
RAG_MCP_BACKEND = "lancedb"

Option B: Standard MCP JSON (.mcp.json or ~/.elpis/mcp.json)

{
  "mcpServers": {
    "rag": {
      "type": "stdio",
      "command": "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python",
      "args": [
        "/home/masih/Desktop/p/rag-mcp-lancedb/server.py"
      ],
      "env": {
        "RAG_MCP_WORKSPACE_ROOT": "/home/masih/Desktop/p",
        "RAG_MCP_BACKEND": "lancedb"
      }
    }
  }
}

Codex

Add to ~/.codex/config.toml:

[mcp_servers.rag]
command = "/home/masih/Desktop/p/rag-mcp-lancedb/.venv/bin/python"
args = ["/home/masih/Desktop/p/rag-mcp-lancedb/server.py"]

[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/home/masih/Desktop/p"
RAG_MCP_BACKEND = "lancedb"

Expected result: the client discovers query_knowledge_base, and a query returns ranked passages with source paths from the requested scope.

What is rag-mcp

rag-mcp is a local hybrid-search MCP server: point it at a file or folder, ask a question in plain language, and get back the passages that actually answer it — with the exact file and location, not a guess.

Coding agents normally search a codebase by opening files one at a time or dumping an entire repository into the conversation. Both waste time and context. rag-mcp replaces that with one tool call: search by meaning, get ranked results with sources, keep going. Embeddings, vector search, and reranking all run locally; the server exposes one read-only MCP tool to compatible clients.

No third-party logo or benchmark — the image above is the actual retrieval-accuracy evidence this repo ships, not decoration.

Retrieval accuracy

Recorded, reproducible retrieval runs live under evals/ — not part of CI, kept as evidence. Every point in the chart is a question-level grade from a recorded query_knowledge_base run, graded against a known answer — not simulated.

The top result is a single real directory containing a 29-file Rust crate, 6 Python scripts, and a Jupyter notebook, searched with doc_path pointed at the whole directory — the server has to find the right file among three languages, not just the right passage in one document. The other five rows are single-document or project-scoped runs: a notebook, two books, a paper, a novel, and the rag-mcp codebase itself.

The chart reports exact full answers separately from answers that were full or partial. The codebase and structured-document rows are easier to retrieve exactly; the long narrative rows contain more interpretive answers spread across passages, which makes chunk-based retrieval less reliable. This is a description of these recorded runs, not a claim about every corpus or a benchmark against other retrieval tools.

Per-question results for every corpus — including where each miss actually failed (vague topical overlap, a truncated chunk, or the fact genuinely absent from top-k).

An earlier, now-superseded Napoleon run with reranking manually disabled scored 93.3% — that number describes raw BM25+vector retrieval in isolation, not this server as it actually runs, and is kept in evals/napoleon/experiment_log.md only for its chunk-size finding (1000 characters beat 300). The earlier separate Rust-only and notebook-only runs are likewise superseded by the combined directory test above and kept under evals/elpis-memories-crate/ and evals/notebook/ only as raw evidence.

This is six recorded corpora at one point in time — not a benchmark against other retrieval tools.

The problem

Coding agents commonly retrieve context by either opening files one by one or loading a large portion of the repository. The first can miss relevant files; the second consumes context with material the current task may not need.

rag-mcp moves retrieval into one local tool call so the agent can search by meaning without making the entire tree part of every prompt.

Architecture

The retrieval pipeline operates across two decoupled stages:

1. Scope Ingestion & Storage: Target files matching the allowlist (.rs, .py, .ts, .md, .ipynb, .pdf) are processed through structure-aware AST chunking (chunk_document), preserving markdown heading hierarchies, code symbol scopes, and notebook cells. Chunks are simultaneously indexed into LanceDB's dense vector table (all-MiniLM-L6-v2) and embedded Tantivy inverted full-text search (FTS) index.

2. Dual Retrieval & Precision Funnel: Each query_knowledge_base invocation executes parallel dense vector cosine search and lexical Tantivy BM25 queries ($top_k=20$). Candidates are unified via Reciprocal Rank Fusion ($\text{RRF}, k=60$), then passed through a neural Cross-Encoder (cross-encoder/ms-marco-MiniLM-L-6-v2) for full-attention relevance re-scoring before delivering ranked source citations.

How it works

Repository structure:

rag-mcp/
├── server.py       # stdio JSON-RPC MCP host
├── rag/            # chunking, BM25, vector search, reranking
└── utils/proxy.py  # local proxy-environment handling

Technical boundaries:

• one MCP tool: query_knowledge_base(query, doc_path?);

• default embeddings: all-MiniLM-L6-v2 (~80MB, fast — overridable, see Configuration);

• reranking runs by default: cross-encoder/ms-marco-MiniLM-L-6-v2 (~80MB, fast — overridable, or disable it entirely);

• local embedded/on-disk Qdrant;

• doc_path can scope each call to a file or directory;

• per-path indexes are persisted under rag/rag_db_v2/;

• common large/build directories such as .git, node_modules, .venv, dist, build, and target are rejected;

• configurable depth/token limits fail explicitly instead of scanning an unbounded tree.

Configuration

Every retrieval knob is an environment variable, not a source edit. Unset means the default shown:

The shipped defaults use the lightweight, fast pair (all-MiniLM-L6-v2 + ms-marco-MiniLM-L-6-v2, ~80MB each) for instant zero-dependency local operation out of the box.

To run the higher-capacity pipeline evaluated in the benchmark suite above, switch to Ollama Qwen embeddings:

[mcp_servers.rag.env]
RAG_MCP_WORKSPACE_ROOT = "/absolute/path/to/your/project"
RAG_MCP_EMBED_PROVIDER = "ollama"
RAG_MCP_EMBED_MODEL = "qwen3-embedding:8b"

openai_compatible is the one non-local option: it leaves the machine. One client implementation covers real OpenAI, Ollama's own /v1 endpoint, and Qwen/DashScope's and Gemini's OpenAI-compatible modes — install the optional openai package (uv sync --extra openai), then set RAG_MCP_EMBED_PROVIDER=openai_compatible, RAG_MCP_EMBED_MODEL to the provider's model name, RAG_MCP_EMBED_API_KEY, and RAG_MCP_EMBED_BASE_URL if the provider isn't OpenAI itself. The client construction is verified; a real embedding call against a paid provider is not — test it against your own key before trusting it. A cross-encoder reranker type named llm also exists in the code but its scoring is unimplemented scaffolding (every passage gets the same score) — do not set it, it does nothing useful.

Current state

Implemented and verified

• MCP initialize → tools/list → tools/call protocol path.

• Read-only query_knowledge_base tool.

• Workspace-root and explicit doc_path scoping.

• Local hybrid retrieval and reranking.

• Guardrails for excluded directories and oversized scopes.

• Text extraction for PDF and Jupyter notebooks, plus local OCR for common image formats and structured extraction for DOCX, PPTX, XLSX, and CSV files.

• End-to-end registration was exercised through a real MCP client during development.

Implemented but not yet covered by the current tests

• The alternative Ollama embedding-provider path in rag/core.py.

• The openai_compatible embedding provider: client construction is verified, a real call against a paid provider is not.

Planned

Nothing is formally tracked yet. Extend it when a concrete retrieval failure or client requirement appears.

Intentionally unsupported

• Hosted/remote vector databases.

• File types outside the extension allowlist in server.py.

• Write/mutation tools; this server is retrieval-only.

What sets this apart

These are design choices, not novelty claims:

• Local retrieval: source files, embeddings, vector search, and reranking stay on the machine.

• Small transport layer: the MCP host uses direct stdio JSON-RPC rather than depending on an MCP SDK.

• Per-call scope: one server can search different files/directories instead of requiring one fixed knowledge base per project.

• Evidence in the response: returned chunks include source paths rather than only synthesized prose.

Evals and test series

The test suite lives under tests/:

python scripts/bootstrap.py

They cover:

• read-only tool annotations;

• default workspace scoping;

• explicit doc_path scoping;

• rejection of excluded directories;

• rejection of depth-limit violations;

• bootstrap prerequisite checks;

• extraction from DOCX, PPTX, XLSX, and CSV files;

• local OCR extraction from a generated image fixture.

Protocol-level check, without another MCP client:

printf '%s\n%s\n%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"query_knowledge_base","arguments":{"query":"how does reciprocal rank fusion combine bm25 and vector results"}}}' \
  | RAG_MCP_WORKSPACE_ROOT="$PWD" .venv/bin/python server.py

A successful self-query should return evidence pointing at the RRF implementation in rag/core.py.

What the tests prove: MCP transport/scoping/guardrail behavior covered by those cases.

What they do not prove: retrieval quality across arbitrary corpora, cross-client compatibility, or superiority to grep/code-search/RAG alternatives.

Example

query_knowledge_base(
  "how does retry backoff work for failed jobs",
  doc_path="codex-rs/memories"
)

The response is intended for the calling agent: ranked source passages it can use as task context rather than a standalone chat answer.

Future development

Keep the surface small. Add capability only when real usage shows a retrieval, compatibility, or performance gap worth testing.

License

MIT — see LICENSE.