# EpsilonGraph

A typed knowledge graph + epistemic ledger over an Obsidian vault, exposed to
coding agents (Claude Code and anything else that speaks MCP).

It answers questions that no RAG system answers:

- What grounds the rule `scaffold split is the default` — and has it been
  tested, in how many projects?
- If I change this note, which other notes become **wrong** (not merely stale)?
- What does the vault itself admit it does not know?
- Give me the 8k-token subset of this 160k-token vault that this task needs.

> [!note] The mathematics is written down
> **Read [`THEORY.md`](THEORY.md) first if you want to know why any of this
> should be believed.** It states what each sub-system *is* mathematically, and
> then says which claims are proved, which are measured, which are mine, and
> which are still guesses.
>
> The short version: blast radius is a **semiring-provenance transitive closure**
> (Green et al., PODS 2007), context packing is **submodular maximization under a
> knapsack constraint** (Sviridenko 2004; Amanatidis et al., *TCS* 1060, 2026),
> and RRF's `k` has a **derived bound** where the practitioner literature has
> only folklore. Writing those three down found four real bugs that reading the
> code did not.
>
> **Correction, 2026-10-12.** A literature check found that the packing
> formulation is **already published** — arXiv:2607.00725 (Jul 2026) casts
> reader-context construction as budgeted *monotone* submodular maximization,
> with the task-level eval this repo lacks. That claim is withdrawn. The
> replacement candidate is a **dependency-closure constraint under a budget**,
> which is measurably outside that literature because closure terms are
> supermodular. `THEORY.md` §3 and §5. **What replaced it, per prior work, is
> the [related-work table](#related-work-the-scope-map) below** — one row per
> work, each naming the quantity it leaves open.

> [!note] The product view
> Evaluating this as a product rather than a research project — install,
> resource auto-acquisition, multi-vault, distribution, scale — is written up
> in **[`PRODUCT.md`](PRODUCT.md)**.

## Related work: the scope map

What each prior work *models*, what it *leaves open*, and whether this project's
measured results contradict it, confirm it, or are **orthogonal** (touch a
different quantity). Every cell is drawn from `RESEARCH.md` §4 and §P8.1 and
`THEORY.md` §5; no citation or number appears here that is not already there.

| prior work | what it models | what it leaves open | this project's result |
| --- | --- | --- | --- |
| **Hwang, *Retrieval-Governed Context*, Zenodo 2026-04-15** | scope gates, priority weights, conflict resolution and **mandatory injection** over a unified instructions+tools corpus; a governance layer | **mandatory injection is "always include this", not "if `X` then `deps(X)`"** — no closure, no graph-theoretic constraint; no theory; and no per-note closed-pack cost, so `{i : h({i}) ≤ B}` does not arise | **orthogonal.** Confirms *why this corpus makes the question non-trivial* (its effect scales with metadata richness) but models no dependency; it supplies the F1 delta for *removing* a gate, not a price for carrying one |
| **arXiv:2607.00725, *What Survives Into Context* (Jul 2026)** | reader-context construction as budgeted **monotone submodular maximization**, with a packer over relevance/coverage/representativeness/diversity and a task-level eval (answer-in-context) | the **supermodularity of the closure terms** — it has no dependency structure at all, so it never meets the fact that both natural closure terms are supermodular; also reports retrieval and answer quality separately, never answer-quality *conditioned on presence* | **contradicts (one assumption) + orthogonal (the rest).** Its `f` assumes **monotone**; a conflict penalty is supermodular, so the `(1−1/e)` guarantee stops applying and the correct citation becomes the non-monotone knapsack bound. Its 14B reversal **bounds this project** |
| **arXiv:2603.03180, *Type-Aware RAG with Dependency Closure* (Mar 2026)** | typed knowledge graph + **minimal dependency closure** over the graph, for **compilability** (missing declarations, type inconsistencies, so a solver can execute the model) | the **budget** — closure is satisfied *fully, then* the model is built, with no packing and no token cost | **orthogonal.** Same phrase, different purpose: its closure is a correctness precondition that is never paid for out of an answer's budget, so it has no `h({i})`, no feasibility frontier, and no supermodularity result |
| **DCTR, *Dual-Constraint Subgraph Optimization for KG-RAG* (AAAI 2026)** | subgraph optimisation for KG-RAG, with dual constraints on the subgraph | **the decomposition, and any closure-cost quantity.** Read at title/abstract level only — this row is the weakest here and is marked as such | **orthogonal (unverified).** On the available text it reports neither `h({i})` nor the `present`/`correct`/`given it` split; a reason to fetch it before submission, not a claim of difference |
| **vendored `facetpack` / ACE-RAG** (`joelvarun/facetpack`; `nayanananto/Diagnostic-for-Budget-Constrained-Multi-Hop-RAG`) | cost-aware **submodular context packing** as a drop-in for top-k; ACE-RAG supplies the published objective and the answer-in-context metric, both ported (`epsilongraph/priorart.py`, `epsilongraph/aic.py`) | **no dependency structure.** The objective is re-measured **monotone and submodular** on our corpus exactly as published, but three of `f`'s five terms do not see the query, and neither implementation prices a dependency | **confirms (the port) + contradicts (as evidence).** The port is faithful (submodularity re-measured, metric matches on 8 cases incl. the adversarial one); but intrinsic objective value is **not** evidence of retrieval quality — on the task proxy the baseline `rank_order` wins every column |

The three quantities this project stands on — `h({i})` / the feasibility
frontier, the supermodularity of the closure terms, and the
`present`/`correct`/`given it` decomposition — are each attributed above to
exactly one prior work that lacks them (`RESEARCH.md` §P8.1).

## Why this is not a RAG project

Measured on the live vault (`/home/hugo/Documents/Obsidian`, 103 notes):

| Measure | Value |
| --- | --- |
| Whole vault | 642,293 chars ≈ **~160k tokens** |
| Rules + graph skeleton (all `[!summary]`/`[!important]`/`[!warning]` + every `Relations` block) | ≈ **~29k tokens = 18%** |
| Typed relation edges | 491 |

Retrieval collapse is documented at the 10K–100K *document* range. At 103 notes
this is **a long-context selection-and-ordering problem, not a recall problem**.
The gap the vault actually has is **epistemic state** — grounding, evidence,
cross-project status, contradictions, admitted gaps, blast radius — and the
vault already models all of it. epsilongraph supplies the missing **interface**.

See [`DESIGN.md`](DESIGN.md) for the full argument, the boundary conditions that
exclude GraphRAG/RAPTOR/semantic chunking at this size, and the explicit
not-doing list.

## Install

No third-party dependencies. Python 3.11+ (stdlib `sqlite3` with FTS5).

```bash
cd ~/codes/EpsilonGraph
python3 -m venv .venv && source .venv/bin/activate   # a venv: Arch's PEP 668 blocks system pip
pip install -e .            # gives you the `epsilongraph` console script
# or, without installing anything:
python3 -m epsilongraph stats
```

Then one command fetches and verifies every external resource:

```bash
epsilongraph acquire status          # what is missing, and the exact fix for each
epsilongraph acquire obsidian-mcp    # checksummed prebuilt Layer 0 binary
epsilongraph acquire model           # ollama server + qwen3.5:4b (pull)
```

Optional adapters for URL/file acquisition (installed into the venv, never
required):

```bash
pip install -e '.[acquire]'          # trafilatura, markitdown, html2text
# or pin-by-pin:
.venv/bin/epsilongraph acquire repo install trafilatura
```

Vault resolution order: `--vault` → `$EPSILONGRAPH_VAULT` → `~/.config/epsilongraph/config.json`
→ Obsidian's own registry (`obsidian.json`, the vault marked `"open": true`) → cwd.
The index lives at `~/.cache/epsilongraph/index.sqlite` — **outside the vault**, so it is
never committed and never syncs. The Layer 0 binary, the OSS clones, fetched
content, and the acquisition ledger live under `~/.cache/epsilongraph/` too.

### Installed state (2026-10-11)

Both MCP servers are registered in `~/.claude.json` at **user** scope and both
report **✓ Connected**:

```
epsilongraph:       python3 -m epsilongraph.mcp_server                                    ✓
obsidian-mcp: /home/hugo/codes/EpsilonGraph/vendor/obsidian-mcp/target/release/...  ✓
```

The skill is installed at `~/.claude/skills/vault-retrieval/`. The config was
backed up to `~/.claude.json.bak-kgraph-<timestamp>` before the change, and the
three pre-existing servers (`github`, `filesystem`, `MiniMax`) are untouched.

`obsidian-mcp` points at the local release build rather than `cargo install`, so
`vendor/` must stay. If you would rather not depend on `vendor/`, run
`cargo install obsidian-mcp` (~31 min) and change the command to `obsidian-mcp`.

**LICENSE:** MIT — added 2026-10-16 when the repo went public.


## Use

```bash
epsilongraph index                       # build; ~0.05s for 103 notes
epsilongraph stats                       # layer / provenance breakdown
epsilongraph skeleton                    # the ~29k-token decision layer (L1)
epsilongraph pack "how should I lay out a new training project" --budget 8000
epsilongraph claim "scaffold split"      # grounding + evidence + projects + contradictions
epsilongraph impact "Principles/Planning Before Execution"   # blast radius
epsilongraph gaps                        # what the vault admits it does not know
epsilongraph search "docking power"
epsilongraph check                       # staleness + integrity
epsilongraph acquire status              # every external resource, with its fix
epsilongraph graph report                # derived-graph connectivity
```

### Progressive disclosure ladder

| Level | Content | Size |
| --- | --- | --- |
| L0 | root note + layer indexes | ~2k tokens |
| L1 | `epsilongraph skeleton` — every rule and relation, no bodies | ~29k tokens |
| L2 | `epsilongraph pack <query>` — the selected full notes | budgeted |
| L3 | the whole vault | ~160k tokens |

`pack` runs three passes over the budget: cores first, then key points and gaps,
then graph context. A tight budget drops **detail**, never candidates — and the
output states what it dropped.

### Derived edges (`--auto`)

Every index now carries a second, *derived* graph next to the curated one
(`graph.py`): `wikilink` / `backlink` from the author's own links, `cotag`,
`couples` (co-citation), and `lexical` similarity. Two rules keep it honest:

- derived edges live in their own table and are never consulted by `impact` /
  `load-bearing` — a guessed edge must not be able to falsify a note;
- retrieval uses them only when asked: `search --auto`, `pack --auto`,
  `neighbors --auto`, ranked *below* every lexical result (`auto:*` is glossed
  with its kind and weight so a reader can tell inference from curation).

```bash
epsilongraph graph build                 # rebuild the live index + derived graph
epsilongraph graph report                # connectivity: components, orphans, kinds
epsilongraph search "packing" --auto     # derived context, opt-in
epsilongraph neighbors "Flow Matching" --auto
```

`graph build --source <any markdown tree>` builds a graph over foreign markdown
(an acquired repo, a fetched inbox) in its own index under
`~/.cache/epsilongraph/graphs/` — the live vault's index is never touched.

### Missing-link suggestions (suggest at most, never auto-write)

`graph suggest` ranks unlinked note pairs by derived evidence (co-citation
first, lexical last), asks the local reader model for one of the eight typed
buckets, and writes a **report** — paste-ready lines for the note's
`## Relations` block. Nothing is ever written into the vault: the typed graph
stays hand-curated, which is the product's core claim about quality.

```bash
epsilongraph graph suggest --top 8 --out suggestions.md
```

## Resource auto-acquisition

Everything this project does not own — the Layer 0 Rust binary, the reader
model, the adapted OSS projects — is fetched, verified, and recorded by one
command. All of it stdlib-only, checksum-verified, and `--dry-run`-able.

```bash
epsilongraph acquire status                 # one screen: what is missing, and the fix
epsilongraph acquire obsidian-mcp           # checksummed prebuilt binary -> ~/.cache/epsilongraph/bin/
epsilongraph acquire model                  # ollama + qwen3.5:4b (starts the server, pulls the model)
epsilongraph acquire repo trafilatura       # pinned-commit clone -> ~/.cache/epsilongraph/repos/
epsilongraph acquire fetch --url https://example.com/doc --to ./inbox
epsilongraph acquire import --path paper.pdf --to ./inbox
```

- the obsidian-mcp download verifies the release's own sha256 digest **before**
  the binary is made executable, and deletes on mismatch;
- repos are pinned by immutable commit SHA and hash-locked in
  `~/.cache/epsilongraph/acquisitions.json` — a later fetch that does not match
  the recorded hash is reported as tampering;
- `fetch` prefers `trafilatura`, `import` prefers `markitdown`; when neither is
  installed the stdlib fallback runs and says which path it took;
- fetched notes carry `acquired: true` provenance so the vault's epistemic
  ledger never mistakes external content for `ours`.

## MCP server

Dependency-free newline-delimited JSON-RPC 2.0 over stdio. Nine tools:

`vault_pack` `vault_claim` `vault_impact` `vault_gaps` `vault_search`
`vault_neighbors` `vault_get` `vault_skeleton` `vault_stats`

```bash
python3 -m epsilongraph.mcp_server --selftest   # drives a real subprocess session
python3 -m epsilongraph.mcp_server              # stdio server
```

Merge `integrations/claude/mcp.json.snippet` into `~/.claude.json`, and append
`integrations/claude/CLAUDE.md.snippet` to your `CLAUDE.md`. The skill at
`integrations/claude/skills/vault-retrieval/SKILL.md` installs to
`~/.claude/skills/vault-retrieval/`.

## Eval

```bash
python3 evals/run_eval.py                                    # dev set (30)
python3 evals/run_eval.py --queries evals/queries-holdout.tsv # held-out (30)
python3 evals/tune_rrf.py                                    # choose k, check on holdout
python3 evals/compare_pack.py                                # 3 packers, paired sign test
python3 evals/compare_packers.py                             # 8 packers x 2 objectives x AiC proxy
python3 evals/closure_gap.py                                 # the §3 measurement, in one run
python3 evals/synth_corpus.py --out /tmp/epsilongraph-synth        # generate an off-vault corpus
PYTHONPATH=vendor/bm25s python3 evals/compare_bm25s.py       # second BM25, cross-check
```

Two 30-query sets with expected note paths, R@1/3/5 + MRR, a four-config
ablation, and a separate coverage metric for graph expansion.

**dev** (`queries.tsv` — where a parameter may be chosen):

```
config        R@1    R@3    R@5     MRR  misses
lexical      0.70   0.97   0.97   0.821  0
+priors      0.70   0.97   0.97   0.821  0
+graph       0.70   0.97   0.97   0.821  0
full         0.70   0.97   0.97   0.821  0
```

**held out** (`queries-holdout.tsv` — where it is checked):

```
lexical      0.80   0.97   0.97   0.871  0
graph expansion:  without graph 0/30 (0%)   with graph 24/30 (80%)
```

**The eval has falsified three of the design's own claims**, each recorded in
the code rather than quietly dropped:

1. Structural priors do not help. They first measured as *harmful* (MRR
   0.817 → 0.767); after the RRF fix they measure as exactly **+0.000 on both
   sets**. Off by default.
2. Graph expansion's MRR delta is exactly 0.000 — because expansion is kept out
   of the ranked list *by construction*, so MRR cannot move. The right metric is
   coverage: 0/30 → 24/30.
3. `RRF_K = 60` was wrong. See below.

### Packing: rank order vs greedy vs Sviridenko

`retrieve.pack` used to fill the budget in rank order, optimising nothing in
particular. `packopt.py` writes the objective down (`THEORY.md` §2) and scores
all three algorithms on it. Untuned weights, budget 1800 tokens:

```
split     algorithm    mean f   vs rank_order   sign test
DEV       rank_order    3.098   --              --
(30 q)    greedy        3.576   +15.4%          20W/3L,  p=0.0005
          sviridenko    3.692   +19.2%          20W/0L,  p=0.0000

HOLDOUT   rank_order    3.167   --              --
(30 q)    greedy        3.766   +18.9%          27W/1L,  p=0.0000
          sviridenko    3.813   +20.4%          28W/0L,  p=0.0000
```

Holdout improves *more* than dev, and nothing was tuned. `rank_order` lands
within 4% of `best_single` (3.167 vs 3.041 on holdout) — the old pack was, in
objective terms, "pick the single best note and stop".

> **⚠️ The +20.4% above is not evidence of anything.** It was measured before any
> task-level proxy existed. With one (answer-in-context, ported from the paper
> that owns this formulation), the ordering **reverses**: `rank_order` beats every
> optimizer on `hit@pack`, `rule_coverage` and AiC. The cause is structural —
> three of `f`'s five terms do not see the query. Full table and the fix in
> `THEORY.md` §2; run `python3 evals/compare_packers.py`.

### The dependency-closure gap

> **Where this is going: [`RESEARCH.md`](RESEARCH.md)** — the plan and the
> decision points. The executable version is
> **[`todo.json`](todo.json)** (15 tasks, each with a falsifiable `done_when`,
> all runnable locally: pure Python + `qwen3.5:4b` via ollama).
>
> The gap below is the project's research problem. Short version: **10 of 103
> notes have no dependency-closed pack at all under an 1800-token budget**
> (`h({i})` runs min 17 / median 816 / max 2495 tokens), while at 4000 every one
> of them does — a feasibility threshold, not a binary verdict. Reframed as
> `h(S) = cost_full(S) + cost_skel(closure(S))` it becomes a **monotone
> submodular** constraint (verified exactly), so the problem is a non-monotone
> submodular objective under a **submodular knapsack** — the corner neither half
> of the published literature covers.


The vault's own loading rule ("rules before procedures … you will follow the
steps while violating the intent") is a constraint on *what a pack must
contain*, and no packing objective expresses it. Measured on the 30 holdout
queries: **24 packs contain the note that answers the query, and 0 of those 24
carry its dependencies.**

The obvious fix is unavailable. Both natural closure terms — edges with both
endpoints inside the pack, and edges leaving it — are **supermodular** (worst
violation +2.0), so a closure-aware packer is outside the submodular-packing
literature that the 2026 papers, and this repo, live in. Three mechanisms were
built and all three fail: post-hoc repair reaches 100% closure at **0.00
`hit@pack`**, closure-by-construction at **0.03**, and a soft term is a no-op
(the budget is already spent). Details in `THEORY.md` §3; run
`python3 evals/closure_gap.py`.

### Does it hold off-vault? (`evals/synth_corpus.py`)

Every number above comes from one corpus: the author's own vault, 103 notes,
hand-typed relations. That is the project's weakest flank — a result that holds
only on a graph you built by hand is a case study, and no second *real* vault
has the typed-edge, two-tier structure the question needs. So the second corpus
is generated, with the dependency graph, each note's closure size, and each
query's target **set by construction** rather than observed.

```bash
python3 evals/synth_corpus.py --out /tmp/epsilongraph-synth --notes 120 --depth 3
python3 -m epsilongraph --vault /tmp/epsilongraph-synth index
EPSILONGRAPH_VAULT=/tmp/epsilongraph-synth EPSILONGRAPH_INDEX=/tmp/epsilongraph-synth.sqlite \
    python3 evals/closure_gap.py --queries /tmp/epsilongraph-synth/_queries.tsv
```

It reproduces the vault's three structural facts — the gap (28/30 packs contain
the answer, 15/30 carry its dependencies), both closure terms supermodular at
+2.0, `f` itself exactly submodular — and it turns the vault's *observed*
slot-competition explanation into a **controlled curve**. `--depth` sets closure
size; the closure-requiring packer's `hit@pack` falls monotonically as it rises
while the unclosed packer stays flat:

```
closure median      2      5      6     12     <- --depth 2 3 4 6
amort+closed     1.00   0.90   0.93   0.63    hit@pack
amort+rel        1.00   1.00   1.00   0.90    hit@pack, unclosed
```

Closure stays *affordable* at every depth — 100% of notes are closable at 1800 —
and still costs retrieval. That separation, "affordable" against "free", is what
the live vault could show but not vary. The generator prints a manifest so the
packer's output can be checked against ground truth fixed before it ran.

> Scope, stated so the table is not over-read: the synthetic notes carry planted
> lexical signal and a uniform-depth graph, where the real vault has neither.
> Only the *shape* of the tradeoff transfers across corpora, not absolute R@k.

The reader grid was re-run on this corpus at two closure depths (seed 7, 120
notes, `--depth 2` the control and `--depth 6` the treatment, `qwen3.5:4b`, n=30;
archived to `evals/reader-eval-synth-{d2,d6}-2026-10-12.json`). What transferred
and what did not — one row per headline number, with a verdict — is the table in
[`RESEARCH.md` §P6.3](RESEARCH.md). The short version: the depth-vs-`hit@pack`
curve and the slot-competition mechanism transfer; the *levels* do not, and the
closed-vs-unclosed `present` gap is invisible at depth 2 because a median closure
of 2 leaves nothing to carry.

### What the second BM25 found

`evals/compare_bm25s.py` ranks the same 103 notes with `xhluca/bm25s` — an
independent BM25 on numpy sparse matrices, different tokenizer, different
scoring constants, no field weights. Two independent implementations over one
corpus is the only way to tell *"the ranking is fine"* from *"my ranking agrees
with itself"*.

It agreed on 64% of top-10 sets and disagreed usefully: it exposed a query where
the correct note is ranked **#1 by the title retriever and #18 after fusion**.

The cause was arithmetic, not taste. Score is `1/(k + rank)`. With `k=60` and a
candidate depth of 50, every contribution lands in `[1/110, 1/61]` — under 2×
spread. So appearing in a second list (2×) beat being ranked first (1.8×).
**Ubiquity outranked strength.** `k=60` is tuned for corpora of thousands of
documents; at 103 notes it is a flattening filter.

`k=1` was chosen on dev and confirmed on holdout, where MRR is monotonic in k:

| k | dev MRR | holdout MRR |
| --- | --- | --- |
| 1 | 0.821 | **0.871** |
| 8 | 0.826 | 0.861 |
| 60 | 0.817 | 0.830 |

The split earned its keep: **k=8 scored best on dev and worse on holdout** — it
was fitted to the 30 dev queries and would have shipped as an improvement.
`evals/tune_rrf.py` flags that case as `FITTED` automatically.

With n=30, one query is 0.033 MRR. Differences below ~0.03 are a single query
and are not findings.

## Vendored

Four upstream projects, cloned to `vendor/` and adapted. None is imported by
`epsilongraph/` — the package stays dependency-free; they are measurement and
infrastructure, not runtime dependencies.

| Project | Role | Adaptation | Verified |
| --- | --- | --- | --- |
| [`lstpsche/obsidian-mcp`](https://github.com/lstpsche/obsidian-mcp) | Layer 0, raw vault I/O | built from source (v3.1.0, 29 MB, 30m51s); wired into `integrations/claude/mcp.json.snippet` | live MCP session: `initialize`, `tools/list` → 10 tools under the `read` profile, `search_text "scaffold split"` → `Concepts/Data Splitting and Leakage`, the same note epsilongraph returns |
| [`xhluca/bm25s`](https://github.com/xhluca/bm25s) | independent BM25 | `evals/compare_bm25s.py` — the cross-check that found the RRF bug | runs from the clone with numpy alone; 64% top-10 agreement with epsilongraph, 2 queries only epsilongraph finds, 0 neither |
| [`nayanananto/Diagnostic-for-Budget-Constrained-Multi-Hop-RAG`](https://github.com/nayanananto/Diagnostic-for-Budget-Constrained-Multi-Hop-RAG) (ACE-RAG) | the published objective **and** the task-level metric | `epsilongraph/priorart.py` reimplements `ace_rag/evidence_packer.py`'s objective; `epsilongraph/aic.py` ports `ace_rag/context_quality.py` verbatim | objective re-measured **monotone and submodular** on our corpus (their published claim), so the port is faithful; the metric matches the original on 8 cases including the adversarial bag-of-words one — pinned by `tests/test_closure.py` |
| [`joelvarun/facetpack`](https://github.com/joelvarun/facetpack) | cost-aware submodular packing, a second independent implementation | read, not imported; its single-best guard was compared against ACE-RAG's | its guard compares a *score* against a *sum of scores* — two different quantities — where ACE-RAG's correctly calls its own `_objective_value`. Noted, not copied |

The last two are the 2026-10-12 prior-art adaptation. Their `ace_rag/` cannot run
here (it needs Qwen2.5 3B/7B/14B from HuggingFace, which this machine cannot
reach), so the two pieces that *are* reader-independent were extracted instead:
the objective and the metric. `vendor/` is gitignored except `ace-rag/ace_rag/`,
which `tests/test_closure.py` imports for the parity check.

**One vault-pollution note.** `obsidian-mcp` keeps per-vault state at
`{vault}/.obsidian-mcp`. Setting `OBSIDIAN_MCP_DATA` moves the heavy cache (BM25
index, embeddings) to `~/.cache/obsidian-mcp/vaults/{slug}/`, but the server
still creates a small vault-local `.obsidian-mcp/ignore` by design. It contains
no markdown. `epsilongraph/parse.py` skips the directory anyway — that is insurance
against the coupling, not a fix for a bug that happened.

**Not vendored:** `AnswerDotAI/rerankers` was the right second choice on paper —
cross-encoder reranking is the one technique in the "non-negotiable tier" this
project has not implemented, its core is dependency-free, and its FlashRank
backend is ONNX rather than torch. It was rejected because **its models download
from HuggingFace, which this machine cannot reach** (`curl: (35) Recv failure:
Connection reset by peer`; PyPI is fine). A reranker that cannot load a model is
not a reranker. The design's no-dense conclusion is unchanged either way: at 103
notes with R@5 0.97 there is no measured headroom to win back.



## Tests

```bash
python3 -m unittest discover -s tests
```

126 tests in eleven groups:

- **parity** with the pre-existing `validate_links.py` on the live vault — note
  set, link count, broken-link set, and per-note relation buckets must match
  exactly. Two parsers for one corpus is a drift generator; the agreement is
  *tested*, not asserted.
- **ranking** — pins the three properties that measurement found broken: that
  rank outweighs list membership, that a near-exact title match reaches the top
  5, and that the held-out set is still held out.
- **closure** — three kinds, kept separate on purpose. *Parity*: `epsilongraph/aic.py`
  must agree with the vendored original, and the ported ACE objective must be
  monotone and submodular on our corpus as published. *Properties*: both natural
  closure terms must stay supermodular — if one is ever found submodular,
  `THEORY.md` §3 collapses into §2 and the test says so. *Invariants*: the
  mechanisms return closed, within-budget sets, and the production packer still
  does not.
- **docs consistency** — the numbers the documents agree to repeat are recomputed
  from the repo and compared: the corpus size, the dependency-edge count, the
  test count, and the file layout. Five numbers were corrected in code and left
  stale in prose on 2026-10-12, each found by grepping by hand. This turns that
  into a failure. It deliberately allows a correction note to *name* the value it
  corrects — forbidding that would forbid the retraction itself, which is what
  the first version of the check did.
- **reader harness** — the four claims of `evals/reader.py`, testable without a
  model: that it is stdlib-only (the AST is parsed and the import set checked
  against a whitelist), that the request sends `think: false` (without it the
  model puts everything in `thinking` and every answer scores as wrong), that the
  answer is read from `response`, and that the cache key covers the whole request
  rather than the prompt alone.
- **synthetic corpus** — the artefact assertions of `evals/synth_corpus.py`, the
  off-vault validation. Every other test in this repo assumes the live vault;
  this one asserts that a *generated* corpus actually has typed dependency edges,
  a non-degenerate closure distribution, a depth knob that moves it, and query
  targets that exist in it. It exists because the generator failed in exactly
  that way twice during construction — a bullet-prefixed Relations header and a
  bucket name that was a sentence — and both times it produced a corpus that
  indexed cleanly, looked plausible, and had **zero typed edges**. A closure
  measurement against such a corpus reports that closure costs nothing, and is
  believed.
- **reader report** — the confidence intervals and paired tests of
  `evals/reader_eval.py`'s `report()`, testable without a model. `report()` reads
  an `acc` dict of per-query 0/1 lists, which is exactly what the archived
  `reader-eval-*.json` holds, so the offline path (`--from-json`) is exercised
  both against a hand-built dict and against the real archive. The assertions
  require the Wilson `[k/n]` count and the paired `[n=...]` marker to be present,
  because a p-value without its denominator is the failure this project keeps
  recording — a test that merely checks "a p appears" would pass on the report
  that drops the n.
- **unit** — gloss retention, gap-bullet splitting, the table-escaped-pipe
  contract.
- **acquire** — platform→asset mapping for the Layer 0 download, checksum
  verification (mismatch ⇒ deleted, never kept), tarball extraction and
  ledgering, the stdlib HTML fallback, the one-screen status report, and the
  zero-runtime-dependency invariant (the optional `trafilatura` / `markitdown`
  / `html2text` adapters may only ever be imported inside a function).
- **graph** — the derived-edge computation (exact edge sets, thresholds,
  co-citation, orphan agreement) on a fixture vault; that curated pairs are
  never offered as missing-link suggestions; that a foreign `--source` gets
  its own index under `graphs/` and never resolves the live vault's.

## Layout

```
epsilongraph/
  parse.py       # frontmatter, callouts, glossed relations, gap bullets
  config.py      # vault resolution (5 steps), index location
  index.py       # SQLite + FTS5 build; provenance classification; staleness
  retrieve.py    # 4-retriever RRF, budgeted packing, skeleton
  ledger.py      # claim / impact / gaps — the epistemics
  severity.py    # blast radius as a semiring closure over a severity lattice
  packopt.py     # f as a set function; greedy / Sviridenko; closure machinery
  priorart.py    # the published objective (arXiv:2607.00725), ported
  aic.py         # answer-in-context, ported verbatim — the task-level proxy
  cli.py         # subcommands
  acquire.py     # resource auto-acquisition: Layer 0 binary, ollama model, OSS clones, URL/file fetch
  graph.py       # automatic KG construction: derived edges over any markdown tree, suggestions
  mcp_server.py  # stdio JSON-RPC, 9 tools, --selftest
evals/
  queries.tsv           # 30 dev queries
  queries-holdout.tsv   # 30 held-out queries (do not edit)
  run_eval.py           # R@k / MRR / ablation / graph coverage
  tune_rrf.py           # choose k on dev, check on holdout, flag FITTED rows
  compare_pack.py       # 3 packers, paired sign test -- where THEORY 2's p-values come from
  compare_packers.py    # 8 packers x 2 objectives x the AiC proxy (means, no test)
  stats.py              # the shared statistics: sign test, Wilson + bootstrap CIs, permutation
  closure_gap.py        # the dependency-closure measurement (§3)
  closure_sweep.py      # the depth x budget frontier (P1b)
  synth_corpus.py       # a second, generated corpus with a known dependency graph
  config_manifest.py    # one JSON: vault/index paths, budgets, SEED, reader model, deps
  reader.py             # the local reader: stdlib only, self-starting, cached
  reader_eval.py        # the task-level eval and its grader (P2)
  compare_bm25s.py      # second independent BM25 over the same corpus
  reader-eval-*.json    # the recorded runs, so a number in the docs is traceable
tests/           # parity + ranking + closure + reader + report + docs + unit + synthetic
integrations/    # Claude Code mcp.json / CLAUDE.md / skill
vendor/          # upstream clones — only ace-rag/ace_rag/ is committed
Makefile         # make repro (offline evals -> results/), make repro-reader, make test
```