# docverify-mcp

An MCP server that exposes document extraction **and its deterministic verification** as
tools, so an assistant is structurally unable to report a number that failed its own
tie-out check.

It reads an Ohio black-box test of State cash-basis "Combined Statement of Receipts,
Disbursements and Changes in Fund Balances" (a public record), re-derives every total from
the underlying line items, and only then decides whether it has an answer to give.

## The property

> The model cannot report a number this server has not proven, because the server does not
> give it one.

That is a property of the **server**, not of one tool, and it is a shape rather than a
flag. `extract_statement` returns one of two different objects:

| outcome | shape |
|---|---|
| verified | `verified: true`, and a `line_items` key holding the figures |
| refused | `verified: false`, and **no `line_items` key at all**, at any depth |

Not `line_items: null`. Absent. A caller that reaches for the figures on a refusal gets a
`KeyError`, not a number.

**When a document fails, every tool withholds it — not just the one you asked.** Figures
still appear in a refusal, but only ever as operands of a check that did NOT tie out,
which is the failure diagnosis the spec requires. The operands of checks that *passed* are
withheld too: a check that passes inside a failed document proves internal consistency of
one subset, it does not prove the figure. That holds on the cold path as well, because the
verdict is computed wherever a document is loaded rather than at one call site, and the
session carries the verdict rather than just the document.

Tested in [`tests/test_disclosure.py`](tests/test_disclosure.py), which states the property
and sweeps it across all three tools, every check id in the document, and both the cold and
post-refusal paths; in [`tests/test_contract_shape.py`](tests/test_contract_shape.py),
which walks the payload recursively looking for any figures field; and in
[`tests/test_mcp_stdio.py`](tests/test_mcp_stdio.py), which asserts the same thing about
what actually crosses the wire.

## Tools

| tool | returns |
|---|---|
| `extract_statement(path)` | the structured statement plus `verified` and the check summary, **or** a refusal with the failing checks and no figures. Takes a report **PDF** |
| `verify_totals(path \| statement)` | every foot / crossfoot / articulate check, with the arithmetic shown for everything the server is entitled to disclose |
| `explain_exception(check_id)` | which two numbers disagreed, by how much, and the page each came from |

`extract_statement` takes a report **PDF**, not a pre-extracted `.json`: `verified: true` means
*this server extracted these figures from a document and proved them*, which is a different claim
from *you handed me numbers and they are internally consistent*. `verify_totals` does accept
caller-supplied numbers, because that is honestly what it is — check my arithmetic. Its payload
reports `checks_pass`, never `verified: true`, and carries `claim.kind == "arithmetic_only"`
saying in words that the figures were not extracted by this server.

Three check kinds, all re-derived from the line items:

- **FOOT**: the detail lines in a section sum to that section's reported total, per column.
- **CROSSFOOT**: for every row, the fund-type columns sum to the Combined Total column.
- **ARTICULATE**: net change = receipts - disbursements; ending = beginning + net change.

Each check ties out **to within $1** (`tolerance 1.0`), which absorbs the rounding a published
statement already carries and nothing larger.

On the bundled sample report's FY2024 statement that is **25 checks**, all passing,
relating all 39 `(line item, column)` cells of the 13 line items to something other than
themselves. 25 is not observed, it is **required**: it is what the document's shape says
must run, and fewer is a refusal: the manifest is derived from the document's structure, and a
run producing fewer checks than the manifest refuses instead of reporting a pass.
**Zero checks is never a pass**: a document the server could not derive a manifest from is
refused, not reported as verified with nothing to show.

That report also contains the prior-year comparative on page 17, which is a different
statement of 15 rows and 27 required checks. All three tools take a `page` argument, and
refuse rather than pick one when a file holds more than one statement.
## Install

Fresh machine, clean virtual environment, Python 3.10 or newer (`mcp` 2.x needs 3.10;
developed and run locally on 3.14; the CI matrix covers 3.10 to 3.13):

```bash
cd docverify-mcp                   # your local copy; nothing is published to clone yet

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

pip install -r requirements.txt
pytest
```

Expect **2977 tests: 2976 passed, 1 skipped**, in about six minutes. The skip is one
malformed-input case that has no inline equivalent, and it reports itself as skipped.

Run plain `pytest`, not `pytest -q`: `pytest.ini` already sets `-q`, so a second `-q`
composes to `-qq` and suppresses the pass/fail summary line entirely — you get a wall of
dots, no counts, and no way to tell 2100 passed from 3 passed.

`requirements.txt` pins `mcp>=2.0,<3`. The 1.x SDK had a different server API and this
repo will not import against it.

## Reproduce both paths without Claude Desktop

`tools/mcp_client_demo.py` starts the server as a subprocess and drives it with a real MCP
client over real stdio. It exits non-zero if the clean report fails to verify, if the
corrupted one is not refused, or if any tool discloses a figure the refusal withheld — so
it is a check and not only a demo.

```bash
python tools/mcp_client_demo.py             # both paths
python tools/mcp_client_demo.py --happy     # the clean report only
python tools/mcp_client_demo.py --refusal   # the corrupted report only
python tools/mcp_client_demo.py --path other.pdf --page 10
```

Both bundled reports contain two statements, so the demo names `page=10` on every call;
a file holding more than one statement is refused rather than guessed at.

The happy path reports `verified: true`, 25 of 25 checks passed, and 13 line items. The
refusal path reports `verified: false`, `line_items` absent, and two failed checks:
`FOOT.disbursements.special_revenue` and `CROSSFOOT.public_works`.

## Use it from Claude Desktop

The config file lives at `%APPDATA%\Claude\claude_desktop_config.json` on Windows and
`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS. **It often
does not exist yet** — Claude Desktop only writes it once you have used
Settings -> Developer -> Edit Config, so create it if it is missing. If it already exists and already has an
`mcpServers` object, **merge** the `"docverify"` entry into it; replacing the whole object
silently removes every other server you had configured.

Replace `ABSOLUTE_PATH_TO_REPO` with the full path to your copy.

**Windows** (backslashes must be doubled inside JSON):

```json
{
  "mcpServers": {
    "docverify": {
      "command": "ABSOLUTE_PATH_TO_REPO\\.venv\\Scripts\\python.exe",
      "args": ["ABSOLUTE_PATH_TO_REPO\\run_server.py"]
    }
  }
}
```

**macOS / Linux**:

```json
{
  "mcpServers": {
    "docverify": {
      "command": "ABSOLUTE_PATH_TO_REPO/.venv/bin/python",
      "args": ["ABSOLUTE_PATH_TO_REPO/run_server.py"]
    }
  }
}
```

Restart Claude Desktop fully (quit it, do not just close the window), then ask it to
extract `ABSOLUTE_PATH_TO_REPO/data/sample_report.pdf`, and then
`ABSOLUTE_PATH_TO_REPO/data/corrupted_report.pdf`.

If it does not connect, the logs are the fastest way to find out why:

| platform | log |
|---|---|
| Windows | `%APPDATA%\Claude\logs\mcp-server-docverify.log` and `mcp.log` |
| macOS | `~/Library/Logs/Claude/mcp-server-docverify.log` and `mcp.log` |

The usual causes are a `command` that is not an absolute path to a Python that has the
requirements installed, single backslashes in the Windows JSON, and a config file that is
not valid JSON after hand-editing.

`run_server.py` puts the repo root on `sys.path` itself, so there is no working-directory
assumption and the package does not need to be installed. That is checked by
`tests/test_mcp_stdio.py`, which launches it from an unrelated working directory.

## What the refusal looks like

`data/corrupted_report.pdf` is the real report with one figure changed **inside the PDF**.
`tools/make_corrupt_pdf.py` reads the page's content stream, maps the embedded Type0 font's
CIDs back to characters, and overwrites the seven glyphs of `150,717` in the Special
Revenue column with `150,171`. Same glyph count, and Arial's digits share one advance
width, so nothing about the layout moves. The extractor then reads a genuinely wrong number
out of a genuinely valid PDF.

Every block in this section is printed verbatim by `python tools/mcp_client_demo.py`,
line for line, and `tests/test_readme_claims.py` runs the demo and checks that.

```
verified          : False
line_items key    : ABSENT (refused)
reason            : verification_failed
message           : Refused to return figures: 2 of 25 tie-out check(s) failed: FOOT.disbursements.special_revenue, CROSSFOOT.public_works. The extracted statement is not proven, so no line-item values are returned. The failing arithmetic, if any, is below.
  [x] FOOT.disbursements.special_revenue: 42,423 + 56,082 + 150,171 + 0 = 248,676   vs reported 249,222   delta -546
  [x] CROSSFOOT.public_works: 0 + 150,171 = 150,171   vs reported 150,717   delta -546
```

and `explain_exception("CROSSFOOT.public_works")`:

```
  DISAGREES: computed 150,171 vs reported 150,717, difference -546 (tolerance 1.0)
  computed 150,171 from page(s) [10]
    0 + 150,171
  reported 150,717 on page 10 (Public Works, Combined Total)
```

Then, in the same session, with no path argument and nothing else changed:

```
verify_totals()   : 2 check(s) disclosed, 23 withheld
                    This document did not verify, so disclosure is scoped to the failure diagnosis: the arithmetic o...
explain_exception('FOOT.receipts.general') -> disclosed: False
                    explanation key: ABSENT (withheld)
cells reachable   : 7 (the failing checks' operands), 0 beyond the refusal
```

That last block is the part that took two rounds to get right, and it is the reason the
property is stated about the server rather than about `extract_statement`.

Rebuild the fixture at any time with `python tools/make_corrupt_pdf.py --verify`.

## Testing

```bash
pytest                             # the suite
python prove_checks_can_fail.py    # break each guarantee, require red, restore, require green
```

`tests/test_input_envelope.py` runs each malformed-input CLASS through each tool — unusable
PDFs, unusable paths, and 22 shapes of malformed JSON — and requires a structured refusal that
names the offending line item and column, never a bare exception.

**Forty-five breaks.** That is what `prove_checks_can_fail.py` performs: it breaks each
guarantee in turn, requires the tests to go red, restores, and requires green again. A suite
that cannot go red is not evidence, so this runs in CI beside the suite itself.

CI runs the suite and then runs that prover.

## Layout

| path | what it is |
|---|---|
| `docverify/extract.py` | from upstream, with TWO marked divergences: it records ambiguous cells, and its page geometry is derived and self-checking rather than three constants |
| `docverify/anchors.py` | new: the five anchor rows and the two section selectors, in ONE place for the precondition, the structured checks and the vendored validator |
| `docverify/validate.py` | **vendored** from upstream: the authoritative tie-out checks (anchor patterns imported, not re-written) |
| `docverify/identity.py` | new: ONE canonical identity for a name, and the injectivity rule its check ids must obey |
| `docverify/checks.py` | new: the same checks as structured records, plus the relation-coverage gate |
| `docverify/structure.py` | new: the structural preconditions, the expected check manifest, label normalization, statement-page detection |
| `docverify/prose.py` | new: every verdict sentence, and the one quoting helper document text enters a sentence through |
| `docverify/inputs.py` | new: the input boundary — every input becomes a statement or an `InputError` |
| `docverify/contract.py` | new: the disclosure decision, the verdict, the payload shapes |
| `docverify/server.py` | new: the MCP server, stdio transport |
| `run_server.py` | launcher a client points at |
| `tools/make_corrupt_pdf.py` | builds the corrupted fixture by editing the PDF content stream |
| `tools/make_statement_pdf.py` | synthesizes born-digital Combined Statement PDFs (reportlab) |
| `tools/mcp_client_demo.py` | real MCP client over stdio, drives both paths |
| `prove_checks_can_fail.py` | breaks each guarantee, requires red, restores, requires green |
| `tests/test_structure.py` | the perturbation family, the manifest, the preconditions |
| `tests/test_prose.py` | no payload claims a verification the verdict does not support |
| `tests/test_identity.py` | the twin family: same name and refused, or different names and different ids |
| `tests/test_confusables.py` | the confusable family: mixed script and shared skeleton, both axes, both input paths |
| `tests/test_ambiguous_cells.py` | the rendering family: the printed figure exactly, or a refusal |
| `tests/test_column_shapes.py` | the column-shape family: no finding that is not an error |
| `tests/test_manifest_anchor.py` | the manifest names every expected check exactly once |
| `tests/test_caller_text.py` | the caller's arguments are data; an unknown id answers `found: false` |
| `tests/test_prose_injection.py` | engineered claim strings in every document-supplied field |
| `tests/test_disclosure.py` | the server-wide property, over the whole call surface |
| `tests/test_claim_separation.py` | the two claims, and the relation-coverage gate |
| `tests/test_input_envelope.py` | every malformed-input class, through every tool |
| `tests/test_roundtrip.py` | the output round-trips as input; zero checks is never a pass |
| `tests/test_refusal.py` | the refusal fires, because the input changed |
| `tests/test_contract_shape.py` | the payload shapes, and the guards' teeth |
| `tests/test_parity.py` | the structured checks cannot drift from `validate.py` |
| `tests/test_golden.py` | the extraction, pinned figure for figure |
| `tests/test_mcp_stdio.py` | it serves, over real stdio |
| `claude-pack/` | a Claude Code skill and two slash commands, plus their installer |
| `data/sample_report.pdf` | the public AOS report |
| `data/corrupted_report.pdf` | the same report with one figure overwritten in the PDF |
| `data/expected_golden.json` | the pinned extraction |
| `.github/workflows/ci.yml` | the suite and the prover |

## Claude Code pack

A skill and two slash commands, written for this repo:

```bash
python claude-pack/install_pack.py            # into ./.claude here
python claude-pack/install_pack.py --user     # into ~/.claude everywhere
python claude-pack/install_pack.py --list     # dry run
```

`/verify-report <path>` and `/explain-exception <check_id>`, plus a `tie-out-review` skill
whose entire job is the one rule: never state a figure that did not arrive under
`verified: true`. See [`claude-pack/README.md`](claude-pack/README.md).

## Provenance and license

Extraction and validation are reused, not re-derived. `docverify/extract.py` and
`docverify/validate.py` are vendored from
[ethan-haas/financial-statement-extractor](https://github.com/ethan-haas/financial-statement-extractor)
at commit `6ffac5b`, together with `data/sample_report.pdf` and
`data/expected_golden.json`. Same author, same MIT license.

Precisely: `docverify/validate.py` is **content-identical to upstream modulo line endings, the
provenance header, and one import line** (`import extract` -> `from . import extract`, for
packaging).

`docverify/extract.py` **is not**, and it has now diverged twice. Both divergences are
marked with banners at the top of the file together with their reproductions.

* `_ambiguous_cells` was added, listing every (row, column) that received more than one
  value token, without changing any value the module produced.
* **The page geometry was replaced.** This is a rewrite, not an addition. Upstream's
  absolute `_LABEL_MAX_X = 295` cutoff, its `Column %d` fallback and its 50pt header window
  are **gone**, replaced by a derived, self-checking column model: the extractor establishes
  the column boundaries from the page it is reading and verifies them, rather than trusting
  three constants that happen to fit one layout.

What is still upstream's, unchanged: the token grammar (`_NUM`, `parse_amount`,
`is_value_token`), the row grouping, the 25pt column-clustering gap, the statement-page
test, the header-metadata regexes, the section rule's receipts/disbursements vocabulary, and
**the answer on the bundled report** — `data/expected_golden.json` is byte-for-byte what the
new extractor produces, which is the strongest statement available here and the one
`tests/test_golden.py` and `tests/test_parity.py` make. Earlier versions of this README described
both files as vendored unchanged; that is no longer true of `extract.py`, and this
paragraph is the correction.

`docverify/validate.py` has one small divergence of its own: its five anchor
patterns and its two section selectors were literals, and copies of the same literals lived
in `docverify/structure.py` and `docverify/checks.py`. Three copies of one predicate is a defect waiting for the
vocabulary to widen, which it did. They now live in `docverify/anchors.py` and all three modules import them. The
behaviour on the bundled report is unchanged and `tests/test_parity.py` is what says so.

The line-ending caveat is real and worth stating: the blobs
committed here are LF, but this repository was authored on Windows with
`core.autocrlf=true`, so a Windows checkout materialises them as CRLF and a byte-for-byte
diff against upstream on such a checkout will differ on every line. The two data files are
byte-identical to upstream. New code here is the MCP layer, the structured check records,
the input boundary, the disclosure contract, the corruption tool and the tests.

The sample report is a public record published by the Ohio black-box test of State. Nothing in
this repo is client work.

MIT. See `LICENSE`.