# Sanad MCP server

A scholar-approved Islamic corpus served to any MCP client, with refusal enforced
**server-side**. A prompt can be talked around; a server that returns no passages cannot.

## Quick start

```bash
npm install
node ingest.mjs        # ~9s, builds corpus.generated.json (9,209 passages)
node eval.mjs          # 52/52, 0 fabrications
node smoke-test.mjs    # end-to-end check
node review-queue.mjs  # -> review-queue.md, the passages a reviewer must sign
```

`ingest.mjs` is not optional and its output is not committed — see *Editions and
copyright* below. Every entry point checks for the built corpus first and tells you
so in one line; none of them will hand you a stack trace for a step you skipped.

Then point Claude Desktop at it — Settings → Developer → Edit Config:

```json
{ "mcpServers": { "sanad": { "command": "node", "args": ["C:/path/to/sanad-mcp/server.mjs"] } } }
```

Forward slashes, absolute path. Or from a terminal: `claude mcp add sanad -- node ./server.mjs`

## What's in the corpus after ingest

| Source | Passages | Grading |
|---|---|---|
| Qur'an (Uthmani + Pickthall) | 6,236 | Qur'an |
| Sahih al-Bukhari | 1,278 | Sahih (by collection) |
| Sahih Muslim | 1,281 | Sahih (by collection) |
| Muwatta Malik | 414 | *none attributed* |

Gradings are recorded exactly as the upstream dataset attributes them. Bukhari and
Muslim are marked "Sahih (**by collection**)" — inherited from the collection, not an
individual isnad grading, and the server never presents it as one. Malik carries no
grading because the dataset provides none, and inventing one would be the exact
failure this project exists to prevent.

Every passage lands `review_status: "unreviewed"`. `reviewed_by` stays `null` until a
named scholar signs off, and `corpus_inventory` reports that honestly.

## Editions and copyright

`sources.json` controls which editions get ingested. The default is **Pickthall**
(d. 1936), public domain in Australia under life + 70.

**Do not switch to `en.sahih` (Saheeh International), `en.asad`, `en.hilali` or
`en.itani` and then redistribute the corpus.** Those are under active copyright.
Yusuf Ali (d. 1953) entered Australian public domain in 2024. The pipeline ships
without a generated corpus for this reason — you build it locally against whichever
edition your scholar names.

## The eval harness

`evalset.json` holds 52 labelled cases. Each declares the verdict the system **must**
return. This is the artefact the scholar contributes to.

```bash
node eval.mjs            # run the suite
node eval.mjs --sweep    # grid-search the gates
node eval.mjs --verbose  # show why each failure failed
```

Two error types, reported separately because they are not equally bad:

- **Fabrication** — answered when it should have refused. Exit code 1. Ship-blocking.
- **Over-refusal** — refused when the corpus did cover it. Merely unhelpful.

Current: **52/52, 0 fabrications, 0 over-refusals.**

The last over-refusal to fall was not a retrieval problem at all — see *The corpus
speaks 1930 English* below. Zero over-refusals against 52 cases is not the same as
zero over-refusals; it means the eval set has stopped finding them, which is a
reason to write harder cases, not to stop looking. Six of the current cases were
written specifically to fail, and four of them did — see *The corpus speaks 1930
English*.

## Three gates, and which one matters

| Gate | Value | What it does |
|---|---|---|
| `retrieval_threshold` | 0.15 | Absolute BM25. **Currently near-inert.** |
| `min_coverage` | 0.35 | Share of IDF-weighted query terms the passage accounts for |
| `max_oov_ratio` | 0.25 | Query weight sitting in terms absent from the corpus entirely |
| `require_pivot_term` | **false** | Tried, measured, unproven. See below. |

A fourth input, `lexicon.json`, sits upstream of all three: it decides which query
words the corpus is even *able* to express. See below.

The sweep showed BM25 score barely affects outcomes — rows are near-identical from
0.15 to 0.45. Coverage and OOV do the work. This is written down rather than tuned
away, because a knob that does nothing is worth knowing about.

`max_oov_ratio` is the one that makes refusal survive scale. A query containing
"Kuraby" or "al-Ghazali" — proper nouns the corpus has never seen — cannot clear it,
however many common words sit beside them.

**Gates are corpus-dependent.** Re-run `--sweep` after any material corpus change.
`corpus.sample.json` (the original 18-passage fixture) over-refuses under these
settings, which is the point: thresholds calibrated for 9k do not transfer to 18.

## Three bugs that only appeared at real scale

**`corpus_inventory` returned every passage id.** Fine at 18. At 9,209 it exceeded the
1MB tool-result limit and the call failed outright. Now returns counts plus a 15-row
sample; the full list lives on the `sanad://corpus` resource.

**Apostrophes split "Qur'an" into `qur` + `an`.** The query term `quran` then matched
no Qur'anic verse at all — but it did match a hadith whose English contains "Glorious
Quran". Asked "what does the Quran say about orphans", the server returned a hadith
about reciting Surah Qaf in the morning prayer.

**No stemming.** Bukhari 1 says "intentions"; the query says "intention". Different
tokens, no match. Both sides are now stemmed with a conservative rule set.

Fixing those three took the orphans query from a prayer-timing hadith to Q 90:15,
Q 107:2 and Q 93:9 — all genuine orphan verses.

## The corpus speaks 1930 English

Pickthall does not contain the word "gambling". Not once in 6,236 verses. He renders
*maysir* as "games of chance" and *khamr* as "strong drink".

So "What does the Quran say about wine and gambling?" hit the OOV gate — `gambling`
has document frequency 0, was charged maximum IDF, and drove the out-of-vocabulary
ratio to 0.45 against a 0.25 ceiling. The server refused, citing no source, while
Q 2:219, Q 5:90 and Q 5:91 sat in the corpus saying exactly that.

The gate was not wrong about the arithmetic. It was wrong about what an unknown term
*means*. "Kuraby" is a subject the corpus does not cover. "Gambling" is a subject it
covers under different words. Treating those two identically is a false negative
wearing the costume of integrity — and on a corpus of religious translation it is
the common case, not the edge case.

`lexicon.json` is the bridge: modern query word to the phrasing this edition
actually uses. Three rules keep it honest.

- **Whole phrase or nothing.** "strong" alone is not "strong drink". Every word of
  the phrase must be in the passage.
- **A phrase must exist in the corpus.** An alias whose words the corpus has never
  seen is not usable, and the term stays out of vocabulary.
- **It is a translation bridge, not a synonym table.** An aliased term is weighted by
  the phrase's own distinctiveness, not by max IDF — but it is still weighted, and it
  can still fail coverage. What must never go in the file is an entry asserting that
  two *concepts* are the same (`interest` -> `usury`, `umrah` -> `pilgrimage`). That
  is a scholarly judgement, and the tokeniser has no business making it.

The bridge is query-side only. It changes which passages are found; it never changes
what a passage says.

It is edition-specific by construction — Yusuf Ali does not write "strong drink" —
so `lexicon.json` records the edition it was written against and the server prints a
warning at startup if `sources.json` names a different one.

Case `n15` guards the boundary: *"What did Sheikh al-Qaradawi rule about gambling?"*
must still return `no_source`. `gambling` is now expressible; `al-Qaradawi` is not,
and that alone must be enough to refuse. `n16` and `n17` do the same for `pork`.

### The second sweep, and the two bridges that were thrown away

A sweep of ~250 modern query words against the built corpus found 90-odd with document
frequency zero. Nearly all of them — *terrorism*, *mortgage*, *vaccination*, *horoscope*,
*passport* — are subjects this corpus genuinely does not cover, and refusing them is the
system working. Six looked like translation gaps and were built as bridges:

| Query word | Pickthall's wording | Verdict |
|---|---|---|
| backbiting | "backbite" (Q 49:12) | **kept** |
| pork | "swineflesh" (Q 2:173, 6:145, 16:115) | **kept** |
| afterlife | "the Hereafter" | **kept** |
| idolatry | "idolater", "idol" | **kept** |
| slavery | "slave", "bondmen" | **dropped** |
| sorcery | "magic" | **dropped** |

The last two passed the eval set and were still wrong. Pickthall's "slave" and "bondmen"
almost always render *'abd* in the sense of a servant of God, so *"What does the Quran say
about slavery?"* returned Q 42:27 ("His slaves would surely rebel") and a Malik hadith
about prayer. His "magic" is overwhelmingly what disbelievers call revelation — *"this is
mere magic"* — not *sihr* as a practice. Both bridges were polysemous, not archaic, and a
bridge that lands on the wrong sense of a word is worse than a refusal: it is confident.

`review-queue.mjs` is what caught them. The eval set only checks that *a* passage came
back containing *a* word; reading the passages is what showed they were about something
else. Both are recorded in `SCHOLAR-REVIEW.md` as rejected candidates, with the verses
that condemned them, so nobody re-proposes them from the word list alone.

## Source names are not subjects

*"What does the Quran say about orphans?"* is a question about orphans. "Quran" says where
to look, not what about — but it is a content token like any other, rare enough to carry
real IDF, and a few verses happen to contain it. Left alone it pulled Q 41:26 (*"Heed not
this Qur'an"*) and Q 43:31 (*"If only this Qur'an had been revealed..."*) to the top of
almost every question phrased that way, whatever the subject. Four of the eval cases were
passing on their third and fourth result.

`SOURCE_TERMS` — quran, hadith, sunnah, surah, ayah — now rank at a discount and earn no
coverage: a passage does not become responsive by containing the word "Quran". They are
demoted rather than dropped, so a query that is *only* a source name still has something
to score.

It cost something. The coverage gate used to pass every case at 0.45 as well as 0.35;
it no longer does, because the noise that was inflating rankings was inflating coverage
too. That trade is written into `policy.json` rather than tuned away.

## A gate that did not earn its place

`require_pivot_term` demands that the most distinctive query term appear in a passage.
It was written to fix the orphans problem. Once tokenisation and stemming were
corrected it made **no measurable difference** — 43/44 either way, identical rankings
on the queries that motivated it.

It is off by default and kept behind a flag rather than deleted, so the idea is not
re-invented. Revisit if the eval set grows and a case appears that needs it. The
lesson is that the gate was treating a symptom; the bug was in the tokeniser.

## Two findings worth repeating to the scholar

**Retrieval broke silently when the corpus grew.** At 18 passages, refusal worked. At
9,209, "When is the mosque in Kuraby open?" started returning confident answers,
because scores were normalised against the best hit — so the top result always looked
like a match. Nothing about the demo would have revealed this. Only the eval set did.

**One apparent bug was a mislabelled question.** "What is the Hanafi position on raising
hands in prayer?" was tagged unanswerable, but the corpus *does* hold hadith on raising
hands. What it lacks is the madhhab attribution. Presenting those hadith as "the Hanafi
position" is fabrication of a different kind, so the case moved to the policy gate and a
new rule was written. The eval set improved the policy, not the code.

## Tools

| Tool | Purpose |
|---|---|
| `answer_question` | Main path. Gate, then retrieve, then verdict. |
| `search_corpus` | Raw retrieval, no policy gate. Not for user-facing questions. |
| `get_passage` | Fetch one passage by id. |
| `check_policy` | Test where the refusal boundary sits. |
| `corpus_inventory` | Coverage and review status, reported honestly. |
| `list_gaps` | Distinct failed questions, ranked by how often asked. The improvement queue. |

## Teaching a child from the corpus

`teach.mjs` is the first agent built on top of the server, and it is the hardest test of
the architecture — because "make this age-appropriate" means rewriting, and rewriting is
where fabrication gets in.

```bash
node teach.mjs "Why do Muslims fast in Ramadan?"
node teach.mjs "..." --html lesson.html
node teach.mjs "..." --model ollama:llama3.2     # a local model writes the intro
node teach.mjs "..." --model fabricate           # hostile stub, to watch the check work
```

**No model runs by default.** The default lesson arranges what the corpus returned — the
Arabic, the translation, the citation, the grading, the review status — adds a *Words to
know* panel and an *Ask your teacher* panel, and paraphrases nothing. For scripture aimed
at children, "we do not rewrite it automatically" is a position, not a shortfall.

*Words to know* is `lexicon.json` turned outward. The bridge exists so a 2026 reader can
find a 1930 translation; the same table, shown to a child, explains why the page says
*strong drink* when everyone at home says alcohol. Morphological pairs are suppressed —
telling a child that "idolatry" is written "idol" teaches nothing.

**When a model is used, every sentence is checked back against the sources.** A sentence
may use only words that appear in the retrieved passages, in the child's own question, or
in `teach-vocab.json` — a short, explicit, scholar-editable list of the words a lesson is
allowed to add. Anything else is dropped and the offending words are printed:

```
✗ The Prophet said this in Sahih al-Tirmidhi number 4127.
    not in any source: prophet, said, sahih, tirmidhi, number, 4127
```

The limit is stated in the code, in the lesson footer, and here, because it matters: this
is **lexical** grounding. It catches a lesson that introduces material the sources do not
contain — an invented name, a fabricated citation, a hadith that does not exist. It cannot
catch a lesson that rearranges real words into a claim the sources never made. It is a
floor under the model, not a replacement for the scholar reading the lesson.

The three verdicts survive the trip. A ruling question produces no lesson and says so; a
question outside the corpus produces no lesson and says so. There is no path where a child
gets a confident guess because the refusal felt unfriendly.

## Running it against a local model

Ollama is not an MCP client, so `agent.mjs` is the bridge: MCP to `server.mjs` on one
side, Ollama's HTTP API on the other, and the grounding check over whatever comes back.

```bash
npm run doctor          # is this machine ready? checks node, corpus, server, Ollama, models
node agent.mjs "Why do Muslims fast in Ramadan?"
node agent.mjs "..." --mode tools --model qwen2.5:7b
```

`npm run doctor` is the first thing to run on a new machine. It checks each part of the
chain in the order it would break, asks the server three real questions covering all three
verdicts, and prints the one command that fixes each failure. A missing local model is a
warning, not a failure — the corpus, the server and the refusals are the demo; a model is a
bonus on top.

**Two modes, and the difference is the point.**

`--mode grounded` (default) asks the server first, always. On `referred` or `no_source`
**the model is never called at all** — there is nothing for it to write from, and inviting
it to try is exactly how a refusal gets softened into a paragraph. Only on `answered` is
the model asked to compose, and only from the passages that came back.

`--mode tools` hands the model the sanad tools and lets it decide. This is the demo worth
watching, because a small model will sometimes skip the tool and answer from memory:

```
$ node agent.mjs "What did Sheikh al-Qaradawi rule about gambling?" --mode tools

  The model never called the tool. It answered from its own memory.
  Nothing was returned by the server, so every content word it used is unsourced.

  3 sentence(s) struck by the grounding check
  ✗ Imam al-Ghazali wrote about this in Ihya Ulum al-Din.
    not in any source: imam, ghazali, wrote, ihya, ulum, din
```

It fails closed. The server handed it nothing, so there was nothing to be grounded in, and
the check struck every sentence. That is the architectural claim being tested by the case
it was built for rather than asserted on a slide.

The grounding check itself now lives in `ground.mjs`, shared by `teach.mjs` and
`agent.mjs` rather than copied into each.

## Files the scholar reviews

- **`sources.json`** — which editions. His call, not a default he inherits.
- **`policy.json`** — nine refusal rules as plain regex with human labels. A draft.
- **`lexicon.json`** — the edition's vocabulary, in his edition's words. Rewrite it if
  he names a different translation.
- **`evalset.json`** — the labelled questions. His to extend.
- **`teach-vocab.json`** — the only words a lesson may use that are not in the sources.
  Short by design: every entry is a word a model is permitted to introduce.
- **`gaps.jsonl`** — written at runtime, every failed question.
- **`SCHOLAR-REVIEW.md`** — the six decisions that are his and not the engineer's,
  each with the evidence behind it. Start here.
- **`review-queue.md`** — regenerated by `node review-queue.mjs`: the 81 passages the
  server has actually surfaced, out of 9,209. Reviewing these is an hour. Reviewing
  the corpus is a year.

He should never need to open `server.mjs`.

## Not yet built

Arabic morphological stemming (currently diacritic stripping only). Embedding
retrieval — which would make most of `lexicon.json` unnecessary, at the cost of an
inspectable failure mode. Per-madhhab tagging. A review UI so approval does not mean
hand-editing JSON. Audio.

The coverage gate no longer has free headroom. As of 2026-09-04 the sweep passes 52/52
at `min_coverage` 0.35 and 51/52 at 0.45 — the case that falls is a06, *"being steadfast
in hardship"*, at coverage 0.375. Tightening the gate now costs a real answer rather than
nothing, which is a more honest picture than the one before: the earlier headroom was
partly source-name noise inflating the numerator.

Left at 0.35, and written down rather than changed quietly — that call belongs to the
scholar, and `SCHOLAR-REVIEW.md` puts it to him in those terms.