claude-rag — MCP RAG Server for Markdown Knowledge Bases

Author: Sergio Angelastro — MIT License

MCP server that indexes a folder of .md files into a local SQLite vector store and exposes hybrid search (BM25 keyword + semantic embeddings) as Claude Code tools. The core RAG system (chunking, embedding, SQLite vector store, MCP server, hybrid search, retrieval eval) is original work by the author.

Hook system (hooks/) inspired by agd-memory by @Pinperepette (MIT) — see CREDITS.md

Architecture

📊 Interactive diagram → · 🔎 How kb_search finds an answer, step by step →

Three components working together:

Stack: Python · fastembed / ONNX Runtime (embedding paraphrase-multilingual-MiniLM-L12-v2 ~120MB + reranker mmarco-mMiniLMv2-L12 int8 ~120MB, CPU-only) · SQLite · MCP stdio

Related MCP server: flightlog

Tools exposed

Setup

1. Clone

# Default layout: repo sits inside the KB folder
# KB files (.md) go in the parent directory
git clone https://github.com/sangelastro/claude-rag ~/.claude/my-kb/rag

Or clone anywhere and point to your KB folder via env var (see step 3).

2. Install dependencies

cd ~/.claude/my-kb/rag
pip install -r requirements.txt

On first run the model (paraphrase-multilingual-MiniLM-L12-v2, ~120MB) is downloaded automatically from HuggingFace. This model supports 50+ languages including Italian natively.

3. Register in Claude Code

Add to ~/.claude.json under mcpServers:

"my-kb": {
  "command": "python",
  "args": ["/absolute/path/to/rag/server.py"],
  "env": {
    "KB_RAG_DIR": "/absolute/path/to/your/kb/folder"
  }
}

• KB_RAG_DIR — folder containing your .md files (default: ../ relative to server.py)

• KB_RAG_DB — SQLite database path (default: kb.db next to server.py)

• CHUNK_MAX_CHARS — max chars per chunk before splitting (default: 400)

• CHUNK_OVERLAP — overlap in chars between consecutive sub-chunks (default: 80)

If the repo is cloned inside the KB folder (as in the example above), both env vars can be omitted.

4. Restart Claude Code

The server starts automatically. On first launch it indexes all .md files in KB_RAG_DIR.

Optional: Claude Code Hooks

Two hooks auto-inject KB context without explicit kb_search calls. Approach inspired by agd-memory (MIT).

Setup hooks

Copy the relevant sections from hooks/hooks_example.json into your ~/.claude/settings.json, replacing the placeholder paths:

{
  "hooks": {
    "SessionStart": [{
      "matcher": "*",
      "hooks": [{"type": "command", "command": "python /absolute/path/to/rag/hooks/kb_session_start.py"}]
    }],
    "UserPromptSubmit": [{
      "matcher": "*",
      "hooks": [{"type": "command", "command": "python /absolute/path/to/rag/hooks/kb_recall.py", "timeout": 5}]
    }]
  }
}

kb_chunks.json (used by kb_recall.py) is auto-generated next to kb.db on every reindex. No model loading in hooks — scoring uses token overlap only.

Hook behaviour is tunable via env vars:

File structure

rag/
├── server.py           # MCP server
├── reranker.py         # Local cross-encoder reranker + calibrated confidence
├── requirements.txt
├── .gitignore
├── README.md
├── hooks/
│   ├── kb_session_start.py   # SessionStart hook
│   ├── kb_recall.py          # UserPromptSubmit hook
│   └── hooks_example.json    # Hook config template
├── eval/
│   ├── eval_retrieval.py       # Retrieval benchmark against a gold set
│   └── gold_set.example.json   # Gold set format (the real one stays local)
├── docs/how_search_works.html  # Interactive walkthrough of the search pipeline
├── architecture.html   # Technical documentation
└── kb_rag_slides.html  # Architecture slide deck

kb.db, kb_chunks.json, eval/gold_set.json and eval/results.json are generated locally and excluded from git: they contain KB content.

How it works

1. Chunking — each .md file is split on ## headers; frontmatter is stripped; sections longer than CHUNK_MAX_CHARS (400) are further split into overlapping sub-chunks with CHUNK_OVERLAP (80) chars of context continuity

2. Embedding — chunks are encoded with paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, 50+ languages)

3. Storage — vectors stored as float32 BLOBs in SQLite + kb_chunks.json for hooks

4. Search — hybrid: BM25 over an in-memory inverted index + cosine similarity in numpy, the two rankings (top 50 each) fused with Reciprocal Rank Fusion; top-K returned. BM25 catches exact identifiers (table names, ports, hostnames) that a 128-token embedding model blurs; embeddings catch paraphrased natural-language questions

5. Rerank + confidence — the top 10 hybrid candidates are rescored by a local multilingual cross-encoder (ONNX, int8, see reranker.py): one forward pass per (query, chunk) pair, no text generated. The top score is mapped to a calibrated probability (Platt scaling) that the answer is among the results; below KB_MIN_CONFIDENCE the tool says so instead of silently returning noise

6. Hooks — keyword scoring on kb_chunks.json (no model), injected before each prompt

7. Invalidation — mtime-based: only modified files are re-indexed on startup

8. Savings tracking — every search records chars served vs full-file baseline in search_stats table; kb_savings() aggregates the cumulative token reduction without re-reading any file

Environment variables

Evaluating retrieval

eval/eval_retrieval.py measures how often the right section comes back, on a gold set of real queries labelled with their correct file + section. It compares semantic, hook lexical, BM25, hybrid, the actual server.rank() and (optionally) a local cross-encoder reranker. It only reads kb.db and never writes to search_stats.

cp eval/gold_set.example.json eval/gold_set.json   # then write queries about your KB
py eval/eval_retrieval.py --no-rerank

Include unanswerable queries ("kind": "neg"): the script also reports whether a score threshold can tell "not in the KB" apart from a real hit. --calibrate fits the Platt parameters of each reranker with 5-fold cross-validation; copy platt_all into reranker.py.

Measured on the author's KB (5,241 chunks, 50 answerable + 7 unanswerable queries). Reranking 10 candidates with mmarco-mminilm takes ~0.4 s on CPU (4 threads; bge-m3 ~6 s for 20). Its calibrated confidence tells "not in the KB" apart with 0.95 accuracy in cross-validation (calibration error 0.07):

Credits

Hook architecture inspired by agd-memory by Pinperepette (MIT License) — in particular the UserPromptSubmit recall pattern and guard rail logic.