claude-rag

Local RAG for Claude Code over a folder of Markdown files.

🇬🇧 English · 🇮🇹 Italiano

claude-rag turns a folder of .md notes into a knowledge base that Claude Code can search. It runs entirely on your machine: no API keys, no cloud, no data leaving the PC.

• Search tool: Claude calls kb_search("…") and gets back the most relevant sections, each with its source file and a confidence score.

• Automatic recall (optional): hooks add matching notes to your prompt before Claude even sees it.

• Measured, not guessed: a retrieval benchmark (eval/) backs every default.

🔎 Interactive walkthrough: how kb_search finds an answer, step by step →

Contents

1. Quick start

2. What Claude can do with it

3. How it works

4. Choosing the embedding model

5. Optional: automatic recall with hooks

6. Configuration

7. Measuring retrieval quality

8. Project layout

9. Credits and license

Related MCP server: flightlog

Quick start

1. Clone the repo inside your KB folder. Your .md files live in the parent folder.

git clone https://github.com/sangelastro/claude-rag ~/.claude/my-kb/rag

2. Install the dependencies.

cd ~/.claude/my-kb/rag
pip install -r requirements.txt

3. Register the server in Claude Code (~/.claude.json, under mcpServers):

"my-kb": {
  "command": "python",
  "args": ["/absolute/path/to/rag/server.py"]
}

If the repo is not inside the KB folder, add "env": {"KB_RAG_DIR": "/absolute/path/to/your/kb"}.

4. Restart Claude Code. On first start the server downloads the embedding model (~120 MB) and indexes every .md file. After that, only changed files are re-indexed.

What Claude can do with it

How it works

The picture above has three parts.

① Indexing, at startup and on kb_reindex()

1. Each .md file is split on its ## sections. Long sections become overlapping chunks (400 characters, 80 shared with the next one). YAML frontmatter is dropped; every chunk keeps a header with its file and section.

2. Every chunk is turned into a vector by the embedding model.

3. Text and vectors go into kb.db (SQLite). The plain text is also exported to kb_chunks.json for the hooks.

4. Only files whose modification time changed are processed again.

② Search, when Claude calls kb_search()

1. BM25 ranks the chunks by exact words: table names, ports, hostnames, error codes.

2. Semantic search ranks them by meaning (cosine similarity between vectors): paraphrases and natural-language questions.

3. The two top-50 lists are merged with Reciprocal Rank Fusion.

4. A local cross-encoder reranker re-reads the best 10 together with the question and puts them in order. Its top score is turned into a calibrated confidence; below 0.5 Claude is told the answer is probably not in the KB.

③ Hooks, optional and automatic: see below.

Choosing the embedding model

Start with MiniLM. Switch to EmbeddingGemma 2 if your questions are often in natural language, or if your notes are long and full of context.

Switching to EmbeddingGemma 2

python -m venv ~/.venvs/embgemma
~/.venvs/embgemma/Scripts/python -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
~/.venvs/embgemma/Scripts/python -m pip install -r requirements-gemma.txt

On Linux and macOS the interpreter is in bin/ instead of Scripts/. Then point the server at it:

"my-kb": {
  "command": "/home/you/.venvs/embgemma/Scripts/python",
  "args": ["/absolute/path/to/rag/server.py"],
  "env": {"KB_EMBED_MODEL": "embeddinggemma-2"}
}

Things to know:

• The model is recorded in kb.db. When it changes, the next start re-embeds the whole KB instead of mixing vectors of different sizes.

• Every Claude Code session starts its own server. Close or reconnect (/mcp → Reconnect) all open sessions after switching; a session still on the old settings would re-index the KB back to the old model.

• The server keeps only the text part of Gemma in memory (271M of 744M parameters) and runs it in fp32 on CPU: bf16 is ~2.5× slower without hardware support, fp16 is not supported by the model.

Optional: automatic recall with hooks

Two Claude Code hooks add KB context without an explicit kb_search call. They use keyword matching on kb_chunks.json: ~10 ms, no model loaded.

Copy the sections of hooks/hooks_example.json into ~/.claude/settings.json and replace the paths:

{
  "hooks": {
    "SessionStart": [{"matcher": "*", "hooks": [
      {"type": "command", "command": "python /absolute/path/to/rag/hooks/kb_session_start.py"}]}],
    "UserPromptSubmit": [{"matcher": "*", "hooks": [
      {"type": "command", "command": "python /absolute/path/to/rag/hooks/kb_recall.py", "timeout": 5}]}]
  }
}

Configuration

All settings are environment variables in the env block of the server (or of the hooks).

Where the KB lives

Indexing and embedding

Search

Hooks

Measuring retrieval quality

eval/eval_retrieval.py checks how often the right section comes back, on a set of real questions labelled with their correct file and section. It compares every stage of the pipeline, only reads kb.db and records nothing.

cp eval/gold_set.example.json eval/gold_set.json   # then write questions about your KB
python eval/eval_retrieval.py              # add --no-rerank to skip the reranker

Add some questions the KB cannot answer ("kind": "neg"): the script also reports whether a score threshold tells them apart. --calibrate fits the reranker's confidence on your own data. KB_EVAL_GOLD and KB_RAG_DB point it to another gold set or database.

Results on the author's KB (6,885 chunks, 48 answerable and 7 unanswerable questions, October 2026). The cells show how often the right section is first / in the top 5, then the MRR:

How each stage got here, release by release, is in the CHANGELOG.

Project layout

rag/
├── server.py                  MCP server: indexing, hybrid search, tools
├── reranker.py                Local cross-encoder and calibrated confidence
├── requirements.txt           Default dependencies (fastembed)
├── requirements-gemma.txt     Extra dependencies for EmbeddingGemma 2
├── hooks/
│   ├── kb_session_start.py    SessionStart hook
│   ├── kb_recall.py           UserPromptSubmit hook
│   └── hooks_example.json     Hook configuration template
├── eval/
│   ├── eval_retrieval.py      Retrieval benchmark
│   └── gold_set.example.json  Gold set format (yours stays local)
└── docs/
    ├── architecture.svg       The diagram above (architecture.it.svg in Italian)
    └── how_search_works.html  Interactive walkthrough of kb_search

kb.db, kb_chunks.json, eval/gold_set.json and eval/results.json are created locally and excluded from git, because they contain your notes.

architecture.html and kb_rag_slides.html are older presentations written for release 1.3: they do not cover EmbeddingGemma 2.

Credits and license

The RAG system (chunking, embedding, SQLite vector store, MCP server, hybrid search, reranking, evaluation) is original work by Sergio Angelastro, released under the MIT License.

The hook system is inspired by agd-memory by @Pinperepette (MIT): see CREDITS.md.