Sanad MCP server

A scholar-approved Islamic corpus served to any MCP client, with refusal enforced server-side. A prompt can be talked around; a server that returns no passages cannot.

Quick start

npm install
node ingest.mjs        # ~6s, builds corpus.generated.json (9,209 passages)
node eval.mjs          # 46/46, 0 fabrications
node smoke-test.mjs    # end-to-end check

Then point Claude Desktop at it — Settings → Developer → Edit Config:

{ "mcpServers": { "sanad": { "command": "node", "args": ["C:/path/to/sanad-mcp/server.mjs"] } } }

Forward slashes, absolute path. Or from a terminal: claude mcp add sanad -- node ./server.mjs

Related MCP server: tero-mcp-lite

What's in the corpus after ingest

Gradings are recorded exactly as the upstream dataset attributes them. Bukhari and Muslim are marked "Sahih (by collection)" — inherited from the collection, not an individual isnad grading, and the server never presents it as one. Malik carries no grading because the dataset provides none, and inventing one would be the exact failure this project exists to prevent.

Every passage lands review_status: "unreviewed". reviewed_by stays null until a named scholar signs off, and corpus_inventory reports that honestly.

Editions and copyright

sources.json controls which editions get ingested. The default is Pickthall (d. 1936), public domain in Australia under life + 70.

Do not switch to en.sahih (Saheeh International), en.asad, en.hilali or en.itani and then redistribute the corpus. Those are under active copyright. Yusuf Ali (d. 1953) entered Australian public domain in 2024. The pipeline ships without a generated corpus for this reason — you build it locally against whichever edition your scholar names.

The eval harness

evalset.json holds 46 labelled cases. Each declares the verdict the system must return. This is the artefact the scholar contributes to.

node eval.mjs            # run the suite
node eval.mjs --sweep    # grid-search the gates
node eval.mjs --verbose  # show why each failure failed

Two error types, reported separately because they are not equally bad:

• Fabrication — answered when it should have refused. Exit code 1. Ship-blocking.

• Over-refusal — refused when the corpus did cover it. Merely unhelpful.

Current: 46/46, 0 fabrications, 0 over-refusals.

The last over-refusal to fall was not a retrieval problem at all — see The corpus speaks 1930 English below. Zero over-refusals against 46 cases is not the same as zero over-refusals; it means the eval set has stopped finding them, which is a reason to write harder cases, not to stop looking.

Three gates, and which one matters

A fourth input, lexicon.json, sits upstream of all three: it decides which query words the corpus is even able to express. See below.

The sweep showed BM25 score barely affects outcomes — rows are near-identical from 0.15 to 0.45. Coverage and OOV do the work. This is written down rather than tuned away, because a knob that does nothing is worth knowing about.

max_oov_ratio is the one that makes refusal survive scale. A query containing "Kuraby" or "al-Ghazali" — proper nouns the corpus has never seen — cannot clear it, however many common words sit beside them.

Gates are corpus-dependent. Re-run --sweep after any material corpus change. corpus.sample.json (the original 18-passage fixture) over-refuses under these settings, which is the point: thresholds calibrated for 9k do not transfer to 18.

Three bugs that only appeared at real scale

corpus_inventory returned every passage id. Fine at 18. At 9,209 it exceeded the 1MB tool-result limit and the call failed outright. Now returns counts plus a 15-row sample; the full list lives on the sanad://corpus resource.

Apostrophes split "Qur'an" into qur + an. The query term quran then matched no Qur'anic verse at all — but it did match a hadith whose English contains "Glorious Quran". Asked "what does the Quran say about orphans", the server returned a hadith about reciting Surah Qaf in the morning prayer.

No stemming. Bukhari 1 says "intentions"; the query says "intention". Different tokens, no match. Both sides are now stemmed with a conservative rule set.

Fixing those three took the orphans query from a prayer-timing hadith to Q 90:15, Q 107:2 and Q 93:9 — all genuine orphan verses.

The corpus speaks 1930 English

Pickthall does not contain the word "gambling". Not once in 6,236 verses. He renders maysir as "games of chance" and khamr as "strong drink".

So "What does the Quran say about wine and gambling?" hit the OOV gate — gambling has document frequency 0, was charged maximum IDF, and drove the out-of-vocabulary ratio to 0.45 against a 0.25 ceiling. The server refused, citing no source, while Q 2:219, Q 5:90 and Q 5:91 sat in the corpus saying exactly that.

The gate was not wrong about the arithmetic. It was wrong about what an unknown term means. "Kuraby" is a subject the corpus does not cover. "Gambling" is a subject it covers under different words. Treating those two identically is a false negative wearing the costume of integrity — and on a corpus of religious translation it is the common case, not the edge case.

lexicon.json is the bridge: modern query word to the phrasing this edition actually uses. Three rules keep it honest.

• Whole phrase or nothing. "strong" alone is not "strong drink". Every word of the phrase must be in the passage.

• A phrase must exist in the corpus. An alias whose words the corpus has never seen is not usable, and the term stays out of vocabulary.

• It is a translation bridge, not a synonym table. An aliased term is weighted by the phrase's own distinctiveness, not by max IDF — but it is still weighted, and it can still fail coverage. What must never go in the file is an entry asserting that two concepts are the same (interest -> usury, umrah -> pilgrimage). That is a scholarly judgement, and the tokeniser has no business making it.

The bridge is query-side only. It changes which passages are found; it never changes what a passage says.

It is edition-specific by construction — Yusuf Ali does not write "strong drink" — so lexicon.json records the edition it was written against and the server prints a warning at startup if sources.json names a different one.

Case n15 guards the boundary: "What did Sheikh al-Qaradawi rule about gambling?" must still return no_source. gambling is now expressible; al-Qaradawi is not, and that alone must be enough to refuse.

A gate that did not earn its place

require_pivot_term demands that the most distinctive query term appear in a passage. It was written to fix the orphans problem. Once tokenisation and stemming were corrected it made no measurable difference — 43/44 either way, identical rankings on the queries that motivated it.

It is off by default and kept behind a flag rather than deleted, so the idea is not re-invented. Revisit if the eval set grows and a case appears that needs it. The lesson is that the gate was treating a symptom; the bug was in the tokeniser.

Two findings worth repeating to the scholar

Retrieval broke silently when the corpus grew. At 18 passages, refusal worked. At 9,209, "When is the mosque in Kuraby open?" started returning confident answers, because scores were normalised against the best hit — so the top result always looked like a match. Nothing about the demo would have revealed this. Only the eval set did.

One apparent bug was a mislabelled question. "What is the Hanafi position on raising hands in prayer?" was tagged unanswerable, but the corpus does hold hadith on raising hands. What it lacks is the madhhab attribution. Presenting those hadith as "the Hanafi position" is fabrication of a different kind, so the case moved to the policy gate and a new rule was written. The eval set improved the policy, not the code.

Tools

Files the scholar reviews

• sources.json — which editions. His call, not a default he inherits.

• policy.json — nine refusal rules as plain regex with human labels. A draft.

• lexicon.json — the edition's vocabulary, in his edition's words. Rewrite it if he names a different translation.

• evalset.json — the labelled questions. His to extend.

• gaps.jsonl — written at runtime, every failed question.

He should never need to open server.mjs.

Not yet built

Arabic morphological stemming (currently diacritic stripping only). Embedding retrieval — which would make most of lexicon.json unnecessary, at the cost of an inspectable failure mode. Per-madhhab tagging. A review UI so approval does not mean hand-editing JSON. Audio.

The coverage gate now has headroom: the sweep passes 46/46 at min_coverage 0.45 as well as 0.35. Tightening it would buy margin against unseen questions at the price of over-refusing ones the eval set has not thought of yet. Left at 0.35, and written down rather than changed quietly — that call belongs to the scholar.