kb-rag — a local, offline knowledge base for papers and source code

Keep a folder of PDFs, notes and source repositories on your own machine, index it once, and let a local language model answer questions from it with citations. Nothing leaves the computer: the model runs in LM Studio (or any OpenAI-compatible server), the index is one sqlite file, and the two are joined by a small MCP server that offers a kb_search tool. Works on an Apple Silicon Mac and on Linux; the network is needed only during installation.

Three clients can use it out of the box: LM Studio's own chat window, VS Code or VSCodium through the Continue extension, and the opencode terminal agent.

What you get

Large language models are good at explaining things and bad at remembering yours. A knowledge base fixes that cheaply. You keep the material in ordinary folders, a script turns it into searchable chunks, and when you ask a question the model first calls kb_search, reads the handful of passages that come back, and answers from them. For source code the passages carry file paths and line ranges, so "see src/main.c lines 120–180" is a real citation.

The primary goal is not a polished product. Rather, we hope it is small enough to read in full — four short Python files — so that when retrieval looks wrong you can see why and tune it.

Related MCP server: loci-mcp

How it works

Two details are worth knowing. First, the two halves of the search matter for different reasons: vectors find concepts phrased differently from the text, and BM25 finds exact identifiers — a function name, a lemma, a flag — that embeddings fumble. Source code needs the second half badly. Second, documents and code are chunked differently. Prose is cut into windows of about 450 tokens with a 60-token overlap; source files are cut on line boundaries, and every chunk begins with its path and line range.

A source repository is not copied into the library. kb add-repo links it in place and the indexer follows the link, skipping .git, node_modules, build directories, lock files, binaries and anything over 2 MB. After you pull new commits, kb reindex --prune re-embeds only the files whose contents changed.

The retrieval models are BAAI/bge-base-en-v1.5 (embeddings, 768-d, ~210 MB quantised) and BAAI/bge-reranker-base (~1.1 GB), run on the CPU through fastembed. Together they take about 1.5 GB of memory while the server is up.

Quick start

git clone https://github.com/potanin/kb-rag ~/kb
cd ~/kb
./install.sh                 # needs uv; makes .venv, installs pinned packages, puts `kb` in ~/bin
kb warm                      # downloads the retrieval models once (~1.3 GB) — the last step that needs the network

kb add ~/papers/*.pdf        # copy PDFs into the library (category "papers")
kb add-repo ~/src/project    # link a source tree, indexed in place (category "code")
kb search "where is the capability derivation tree updated" --k 5

Then start the search server as a service and connect a client:

• macOS: docs/macos.md — Homebrew packages, LM Studio, launchd.

• Linux: docs/linux.md — packages, LM Studio or llama.cpp, systemd.

• Clients: docs/clients.md — LM Studio chat, Continue, opencode.

install.sh is safe to re-run. It insists on uv's own Python build because the Python that Apple ships (and the python.org installer) cannot load sqlite extensions, which the vector search needs; it prints a one-line check so you know the venv is good before anything else happens.

Adding content

A category is just a subfolder of corpus/; papers, notes and code are conventions, not rules, and the search tool can filter by category.

kb add klein2009.pdf                                 # into papers/ (the default)
kb add ~/Documents/reading --category papers         # a whole folder
kb add ~/notes/*.md --category notes
pandoc thesis.docx -o thesis.md && kb add thesis.md --category notes
ocrmypdf scanned.pdf scanned-ocr.pdf && kb add scanned-ocr.pdf   # scans have no text layer otherwise

Source trees are linked rather than copied:

kb add-repo ~/src/seL4 --name sel4            # corpus/code/sel4 -> ~/src/seL4
kb add-repo ~/src/big/lib --name biglib       # a subtree is fine if the whole tree is too much
kb categories                                 # shows which entries are linked repos

The first index of a large repository takes a while — the embedding model runs on the CPU — so we suggest starting with one project. As a rough guide, a repository of a few thousand source files produces tens of thousands of chunks, and we expect that to take somewhere between a few minutes and an hour depending on the machine. Later runs are incremental.

Which files are indexed is decided by the lists at the top of ingest.py: documents (.pdf .md .txt .rst .tex .html ...), source and configuration files (a long list of extensions plus Makefile, CMakeLists.txt, Dockerfile and so on). They are meant to be edited.

Searching from the command line

kb status
kb search "frobnicate_widgets" --category code --k 3
kb search "ownership types and generics" --category papers --no-rerank

Every hit shows the file, the chunk, which half of the search found it (dense, bm25 or both) and the reranker score. Test here before involving a model: if the right passage does not come back from kb search, no prompt will fix it.

The search server

One long-running server serves all clients, so the models load once and stay warm. The platform documents show how to run it as a launchd agent or a systemd user service; by hand it is:

kb serve                                          # foreground, Ctrl-C to stop
curl -s http://127.0.0.1:8765/health              # {"ok":true,"service":"kb"}
~/kb/.venv/bin/python ~/kb/check-mcp.py "frobnicate_widgets"   # lists the tool, runs one search

The server binds to loopback only, has no authentication, and should stay that way. It speaks streamable-http at /mcp, the current MCP transport; if a client turns out to want the older SSE transport, run it with KB_MCP_TRANSPORT=sse and use /sse in that client's URL. It reads the index on every query, so adding documents needs no restart; only edits to the Python files do.

There is also a plain-HTTP POST /grounding route for front ends that cannot call tools: send {"query": "..."} and get back a context block to prepend to the prompt, plus the passages for citations.

Day to day

kb add paper.pdf                      # copy in and index
kb add-repo ~/src/thing               # link and index in place
kb reindex --prune                    # after git pull, or after deleting files by hand
kb list | kb categories | kb size | kb status
kb remove paper.pdf                   # drops the copy and its chunks
kb remove sel4                        # unlinks the repo and drops its chunks; the repo itself is untouched
rm index/kb.db && kb reindex          # rebuild from scratch

Everything lives in this checkout plus two caches under ~/.cache, so a backup is a copy of ~/kb (linked repositories are not inside it).

Tuning

Most knobs are environment variables read by the scripts; set them in ~/bin/kb and in the service definition so the command line and the server agree.

Limits

It does not OCR scans or read .docx (pandoc and ocrmypdf cover those), pypdf is mediocre on two-column papers (text order can interleave columns), and the quality of the final answer is bounded by the chat model you can fit in memory. We suspect the single best upgrade for a paper-heavy library is a better PDF extractor in extract_text(); for a code-heavy one it is a smaller chunk size.

Troubleshooting

• AttributeError: enable_load_extension — the venv was built on a Python without sqlite extension loading. rm -rf .venv && ./install.sh.

• ModuleNotFoundError: No module named 'mcp.server.fastmcp' — mcp 2.x got installed; uv pip install --python .venv/bin/python "mcp>=1.27,<2".

• Models download again — a cache path fell back to the temporary folder, which macOS and many Linux distributions clean out. Check FASTEMBED_CACHE_PATH and TIKTOKEN_CACHE_DIR in ~/bin/kb and in the service; kb size shows where it is looking.

• The client shows no kb tool — the loaded model lacks tool support, the tool is switched off for that chat, or the server is down. curl http://127.0.0.1:8765/health separates the last case.

• A client refuses the /mcp URL — switch the server to SSE and use /sse.

• Port 8765 in use — lsof -i :8765; change KB_MCP_PORT everywhere.

• A repository indexes far more than expected — generated or vendored trees under names not in the skip list. Add them with KB_SKIP_DIRS, or link a subtree instead of the root.

• A file is in the repo but never found — it is binary, over 2 MB, a lock file, or has an extension the indexer does not know; see the lists at the top of ingest.py.

Repository layout

Licence

MIT; see LICENSE.