vault-rag-mcp

An MCP server that answers questions about a folder of markdown notes. It runs keyword search and vector search over the same corpus, fuses the two ranked lists, reranks the shortlist with a cross-encoder, and hands the result to an MCP client as a tool call.

Everything runs on the machine it is installed on. There is no API key, no inference endpoint, and no account. The only network access is a one-time model download from the Hugging Face CDN the first time you index.

The problem

Search over your own notes fails in two distinct ways, and the two are not variations of one problem.

Keyword search — BM25 over an FTS5 index — is exact, fast, and needs no model. It finds the note that contains the words you typed. Ask it compaction stall and it returns the right note in four milliseconds. Ask it why did writes get slow in the middle of the night and it returns nothing at all, because that phrasing shares no rare token with anything you wrote.

Vector search has the opposite profile. It embeds the query and finds notes that sit nearby in the embedding space whether or not the words match, so the midnight question lands on the compaction postmortem. But exact tokens dissolve. Error codes, config keys, version strings and surnames come back surrounded by notes that are merely about the same sort of thing.

The two failure modes are close to independent, which is why running both and fusing them is not a compromise between two mediocre answers. A note ranked highly by both methods is much more likely to be right than one ranked highly by either. You can watch this on the bundled demo vault:

$ npm run search -- "why did writes get slow in the middle of the night"

  keyword (FTS5/BM25)
    (nothing)
  vector (sqlite-vec)
    1. [0.636] knowledge/operations/the-first-question-in-an-incident-is-what-changed.md
    2. [0.599] sessions/orchard/2026-03-05-backpressure-rollout.md
  hybrid (RRF + rerank)
    1. [-8.489] knowledge/operations/the-first-question-in-an-incident-is-what-changed.md
    2. [-9.753] sessions/ferrite/2026-02-03-compaction-stall-postmortem.md

Related MCP server: Inkdex

How it fits together

flowchart TD
    V["markdown vault"] --> CH["chunk on level-two headings<br/>heading prepended before embedding"]
    CH --> DB[("one SQLite file<br/>note rows + FTS5 index + sqlite-vec table")]
    Q["query from an MCP client"] --> K["keyword branch<br/>FTS5 porter, ordered by rank then id"]
    Q --> S["vector branch<br/>384-dim KNN, then structured filters"]
    Q -.->|"off by default"| HY["HyDE<br/>template expansion, no LLM in the process"]
    DB --> K
    DB --> S
    K --> RRF["reciprocal rank fusion<br/>combines by position, discards the scores"]
    S --> RRF
    HY -.-> RRF
    RRF --> RR["cross-encoder reranker<br/>30 candidates, one batched pass, about 450 ms"]
    RR --> REC["recency boost, additive, frozen clock"]
    REC --> OUT["ranked chunks"]

The reranker reorders; it cannot recall. If the right chunk is not in the shortlist, nothing downstream will find it. On the demo vault it currently costs more accuracy than it earns, and the evaluation below says so with numbers.

The retrieval path

One SQLite file holds all of it: the note rows, an FTS5 index over title and body, chunk text, and a sqlite-vec virtual table of 384-dimensional chunk embeddings. One file means one lock, one backup, and no second service to keep in sync with the first.

Chunking splits on level-two headings, not on a fixed token window. An embedding is a summary of whatever text it was given, so a chunk that straddles a topic boundary produces a vector that sits between two topics and near neither. Markdown headings are a segmentation the author already wrote down, and the heading itself is usually the densest line in the chunk, so it gets prepended to the text before embedding.

The keyword branch is FTS5 with the porter tokenizer, ordered by rank, n.id. The secondary key is not decoration — bm25 ties are common, ORDER BY rank alone is not a total order, and a stable sort over an unordered result set means the top hit is decided by the query plan.

The vector branch asks sqlite-vec for the K nearest chunks by L2 distance and then applies structured filters. It applies them after the KNN, because a vec0 table holds only (chunk_id, emb) and has no metadata columns to filter on. That ordering has a sharp edge: filtering an already-truncated top-40 can return empty for a scope that the corpus genuinely covers, which turns a false negative into a false positive. When a path scope is present the code escalates K along a ladder until enough in-scope results survive, the index drains, or the ladder runs out — and it reports which of the three happened, because "nothing matched" and "I stopped looking" are different answers. See src/search/semantic.ts and tests/search-path-scoping.test.ts.

Fusion is reciprocal rank fusion, which combines the two lists by position and discards the scores. This is the point: a BM25 score is unbounded and corpus-dependent, a cosine similarity sits in a narrow band, and normalising the two onto a common scale makes a branch with no good answers look exactly as confident as a branch with a perfect one. Ranks need no calibration. The cost is real — RRF cannot express how much better the top hit is than the second.

The reranker is a cross-encoder that reads the query and one candidate together and produces a relevance score. It cannot be precomputed, so it only ever sees the shortlist: thirty candidates, one batched forward pass, roughly 450 ms on two cores. It is the single most expensive stage and the only one that scales with the shortlist size. It reorders; it cannot recall. If the right chunk is not in the shortlist, no amount of reranking will find it. On the demo vault it currently costs more accuracy than it earns, for a reason worth reading: see the evaluation below.

HyDE is off by default and template-based. The classical technique asks an LLM to write a hypothetical answer and searches with the answer's vector. There is no LLM in this process and adding one would invert the layer relationship, so what ships instead is a deterministic template expansion that pads the query into answer-shaped text and fuses the result as a third branch. It costs about 20 ms warm and helps on vague paraphrased queries.

Recency is an additive boost applied after reranking, with a frozen-clock parameter. The reranker emits signed logits, so a multiplier would flip the sign on poor matches. Any harness comparing hybrid output has to pass now explicitly or it will drift against itself.

Running it

npm install
npm run reindex          # builds data/vault.db from ./vault — about 15s, 33 notes
npm run search           # five demo queries across all three branches
npm run eval             # score four retrieval configurations on 24 labelled queries
npm test

The first run downloads two ONNX models (roughly 350 MB total: BGE-small-en-v1.5 for embeddings, ms-marco-MiniLM-L-6-v2 for reranking) into node_modules/@huggingface/transformers/.cache. Every run after that is offline.

To run it as an MCP server:

npm run build
npm start                # HTTP daemon on :8848, health at /health, MCP at /mcp
VAULT_RAG_TRANSPORT=stdio npm start   # classic one-process-per-client stdio

HTTP is the default because the ONNX models are process-global. One daemon means the models are loaded once for the whole machine instead of once per connected client, which on a small box is the difference between working and swapping.

Five tools are registered: vault_search, vault_read, vault_list, vault_reindex, and vault_delta. The last one is the interesting one — given a batch of candidate items it returns which the vault already covers and which are new, which is what you want before ingesting notes from somewhere else.

Pointing it at your own notes

cp .env.example .env

Set VAULT_PATH to your notes directory and DB_PATH to wherever the index should live, then npm run reindex. Nothing else is coupled to the vault's layout. The indexer reads YAML frontmatter if it is there (type, date, project, status, source, topics) and works without it; templates/** and lint-report-*.md are skipped at scan time; [[wikilinks]] are extracted into a sidecar table and surface as related-note suggestions on the top hit.

The index is a derived artifact. Delete it and rebuild whenever you want.

The demo vault

vault/ holds 33 synthetic notes — 132 chunks — written for this repository. None of it is anyone's real notes. It describes two fictional systems, a time-series store called Ferrite and a job queue called Orchard, plus a set of engineering-practice notes and a handful on baking and running so that topical scoping and filtering visibly do something.

The topical spread is deliberate. A corpus about one subject cannot demonstrate that a path scope or a type filter changes the answer, and it cannot produce the adversarial case in tests/search-path-scoping.test.ts where a query's entire top-40 falls outside the scope being searched.

Also deliberate: three reference pages share a ## Conventions block verbatim. Identical text embeds to an identical vector, so those results carry a real exact-tie group, and the frozen baseline in tests/fixtures/search-baseline.json can therefore detect a tie-ordering regression. A baseline that never sees a tie cannot.

What the retrieval actually scores

The claim that fusion beats its parts is checkable here rather than asserted. One command, about a minute, no API key, and no network once the models are cached:

npm run reindex && npm run eval

eval/queries.json holds 24 queries and, for each, the note or notes that answer it. Nine are phrased in the corpus's own vocabulary, fifteen the way someone who had forgotten the vocabulary would phrase them. They were written by reading all 33 notes, before any configuration was scored, and none was edited afterwards to change a result. eval/baseline.json holds the last run and is committed, so a regression arrives as a diff; npm run eval -- --check writes nothing and exits non-zero when the numbers move.

recall@k is the fraction of a query's labelled notes inside the top k, averaged over queries. MRR@10 is the mean of 1/(rank of the first labelled note). nDCG was left out because the labels are binary and grading them by eye would invent precision the set does not have.

Split by how the query is worded, MRR@10:

Read the keyword row first. It answers every vocabulary query at rank 1 and returns literally nothing on all fifteen paraphrased ones — 0.000, not a low score. That is the argument for the second branch, in one line, and it is why the fusion is not a compromise between two mediocre retrievers.

Fusion holds up. RRF over the two branches beats both of them on every column of the first table, and the split table shows what it costs to get there: 0.944 on the vocabulary half against keyword's perfect 1.000, in exchange for 0.850 instead of 0.000 on the other fifteen. Losing a little where one branch is already perfect is the trade the fusion is for.

The reranker loses on this corpus, and the table says so. Adding the cross-encoder to the fused list drops MRR from 0.885 to 0.799, all of it on paraphrased queries. Five queries where a simpler configuration wins, with the rank of the first correct note:

The cause is visible once you print what the cross-encoder is being handed. It scores title + snippet, and the snippet is at most 240 characters of the best-matching chunk — which for a short note is often the title chunk, so the passage is little more than the note's own title repeated. The model then has almost nothing to judge. On the queries in that table the correct note comes back scored around -11, which is the cross-encoder saying nothing here looks like an answer, and what you are reading as a ranking is an ordering over noise. The last row of the table changes exactly one thing, the passage becomes the note's own text, and MRR goes to 0.922: above fusion alone, and above every other row. So the reranker is not the wrong idea here; the passage it is given is too short.

The fix is deliberately not shipped. It changes what every search returns, and the evidence for it is 24 queries over 33 synthetic notes.

The recency boost, which is on by default, moves exactly one query on this set, and downward by one rank. Too small to conclude anything from, which is itself worth knowing about a default.

What this does not prove

A 33-note synthetic corpus is not evidence about a 3,000-note real one. It shows that the machinery runs, that the two branches fail in the directions the design claims, and that the numbers reproduce. It does not predict the numbers on a real vault, and the error is not symmetric: on a corpus this small almost every note is about something different, which flatters the vector branch. A vault with fifty notes on one subject is a harder ranking problem than anything measured here, and it is the ordinary case in a vault of three thousand.

The labels are binary and mine. On the "bug or the design" query the reranker's top hit is reference/orchard-queue.md, which does state that a full ingress rejects rather than buffers — a defensible answer that the label does not credit. A larger set with graded labels, written by someone who did not build the retriever, would be a better instrument than this one.

Tests

npm run reindex && npm test

74 assertions across 9 files. They need the index built first, because most of them query it; the runner says so rather than failing on an assertion.

npm run eval is deliberately not one of them. It loads both models and takes about a minute, and its output is a measurement to read rather than an assertion to trip. The committed baseline is what turns it into a regression check, on demand, with npm run eval -- --check.

What they cover, and why those things and not others:

Two of these are worth explaining.

model-singleflight asserts on the source text rather than behaviour, because exercising the real loaders would download and hold hundreds of megabytes of weights. The bug it guards is that if (!model) model = await load() is evaluated by every concurrent caller before the first await settles, so N simultaneous first-queries each start their own load. Caching the in-flight promise fixes it. A structural assertion is a weaker test than a behavioural one and it is the one that can run in CI.

search-baseline freezes a set of query results and asserts byte-identity, but only while a corpus fingerprint (note count, chunk count) still matches. Once the vault changes, byte-identity cannot be asserted and is not faked — the suite falls back to asserting determinism, prints that it did, and stays green. A byte-identical assertion against a moving corpus is a nightly false red, and a false red in the guard layer is how guards get deleted.

Layout

Provenance and honesty about origin

This is extracted from a personal notes system that has been running daily since March 2026 against a private vault of roughly four thousand markdown files. The retrieval code is that code, with two hardcoded paths replaced by configuration. What did not come with it: the vault, the nightly extraction pipeline that fed it, the deployment, and a calibration harness whose labelled evaluation set scored private material and could not be published. eval/ is the public replacement for that harness: smaller, over a synthetic corpus, and honest about what it can support.

The numbers left in the comments — a 0.745 novelty threshold, a measured 0.106 overlap between duplicate and novel claims, the language guard that refuses to score Cyrillic text because the embedding model reports language rather than meaning for it — were measured on that private corpus, not on the demo vault. They are kept because a constant traceable to a measurement is more useful than one traceable to taste, and they are labelled so nobody mistakes them for something this repository can reproduce.

License

MIT.