# spanwatch

Reconstructs the reading order of a laid-out document from block bounding boxes,
and serves it over the Model Context Protocol. It answers one question: given
text blocks with positions on a page, what order does a human read them in?

The ordering is computed geometrically, so it works with no model, no API key,
and no network. A language model can be plugged in to refine the order, but it is
strictly optional: if the model is missing, unconfigured, slow, or returns
something invalid, the deterministic result is used instead and every response
records which path ran.

## Why it exists

A plain "sort blocks by (y, x)" reads a two-column page as scrambled rows and
staples a full-width header onto the first column. spanwatch uses recursive
XY-cut: it splits a region at its widest clean whitespace gutter, choosing
horizontal gutters (which separate stacked bands like a header above a body)
before vertical ones (which separate columns). See `document/layout.py`.

## Architecture

Data flows one way: bytes in, ordered blocks out. The transports are thin; the
decisions live in the middle.

```
CLI (cli.py) ─┐                        ┌─ providers/stub.py   (tests, offline)
              ├─ service.run_reconstruct├─ providers/real.py   (Anthropic, opt)
MCP (server.py)┘   │                     └─ providers/base.py   (interface)
                   │
   io.parse_document ─ config.enforce_limits ─ extract.reconstruct ─ layout.reading_order
```

- `layout.py` is the load-bearing algorithm: recursive XY-cut, pure standard
  library, no model.
- `extract.py` runs the deterministic order first, then lets a provider refine
  it, validating the result and falling back on any failure.
- `providers/` is the only place a model is involved, behind one interface with
  a deterministic stub used by the whole test suite.
- `service.py` is the shared request path; `cli.py` and `server.py` are the two
  transports over it.

The design rationale for the contested calls is recorded in `docs/adr/`.

## Input schema

```json
{
  "pages": [
    {
      "number": 1,
      "width": 612,
      "height": 792,
      "blocks": [
        {"id": "b1", "text": "Title",   "bbox": [40, 30, 570, 70]},
        {"id": "b2", "text": "Left...",  "bbox": [40, 90, 300, 400]},
        {"id": "b3", "text": "Right...", "bbox": [320, 90, 570, 400]}
      ]
    }
  ]
}
```

`bbox` is `[x0, y0, x1, y1]` with the origin at the top-left and y increasing
downward. `width`/`height`/`kind` are optional.

## Library use

```python
from document import parse_document, reconstruct

pages = parse_document(doc)              # doc is the dict above
results = reconstruct(pages)             # no provider -> deterministic
for r in results:
    print(r.method, r.order)             # e.g. "deterministic" ['b1','b2','b3']
    print(r.text(pages[r.number - 1]))   # blocks joined in reading order
```

To let a model refine the order, pass a provider:

```python
from document.providers.real import AnthropicProvider
from document.providers.base import ProviderUnavailable

try:
    provider = AnthropicProvider()       # reads ANTHROPIC_API_KEY
except ProviderUnavailable:
    provider = None                      # fall back explicitly
results = reconstruct(pages, provider)
```

## Command line

```
make venv
make install

echo '{"pages":[...]}' | .venv/bin/spanwatch reconstruct --pretty
.venv/bin/spanwatch reconstruct doc.json --provider none
```

`reconstruct` reads a document from a FILE argument or stdin (`-`), writes the
result JSON to stdout, and logs one JSON object per line to stderr. Flags:

- `--pretty` indent the output JSON
- `--min-gap N` override the gutter threshold
- `--provider auto|none|anthropic` override provider selection

Exit codes: `0` success, `1` runtime failure (unreadable file, bad JSON, invalid
document, limit exceeded), `2` usage error.

## Running the MCP server

```
make serve            # or: .venv/bin/spanwatch serve
```

The server speaks MCP over stdio and exposes one tool,
`reconstruct_reading_order(document)`, returning per-page `order`, reconstructed
`text`, the `method` used (`deterministic`, `model`, or
`deterministic-fallback`), timing (`elapsed_ms`), and audit `notes`. Invalid
input comes back as `{"error": ...}` rather than crashing the server.

The provider is chosen at startup: the Anthropic provider if `ANTHROPIC_API_KEY`
is set and the `anthropic` package is installed (`pip install '.[real]'`),
otherwise none. With no key the server runs fully offline.

## Configuration

All settings come from the environment; CLI flags override them.

| Variable | Default | Meaning |
| --- | --- | --- |
| `SPANWATCH_PROVIDER` | `auto` | `auto`, `none`, or `anthropic` |
| `SPANWATCH_MODEL` | `claude-3-5-sonnet-latest` | model id for the real provider |
| `SPANWATCH_TIMEOUT` | `30` | model request timeout, seconds |
| `SPANWATCH_MIN_GAP` | (per page) | gutter threshold override |
| `SPANWATCH_MAX_PAGES` | `500` | reject documents with more pages |
| `SPANWATCH_MAX_BLOCKS_PER_PAGE` | `5000` | reject pages with more blocks |
| `SPANWATCH_LOG_LEVEL` | `INFO` | stderr log level |

Limits are enforced before any ordering work so oversized untrusted input fails
fast with a clear message instead of exhausting memory.

## Tests

```
make test             # or: .venv/bin/python -m pytest -q
```

The suite runs offline with no API key. It covers the XY-cut cases (single
column, two columns, header-over-columns, sub-threshold gutters), the fallback
contract (unavailable provider, invalid permutation, provider exception), input
validation, and that the model path is never taken without credentials.

Override the interpreter for any target with `make test PY=/path/to/python`.

## Limits

- Reading order assumes left-to-right, top-to-bottom scripts. RTL and vertical
  scripts are not handled.
- XY-cut needs a clean rectangular gutter to split. Overlapping boxes or complex
  magazine layouts with L-shaped text flow can defeat it; the leaf case then
  falls back to a (y, x) sort within the unsplittable region. TODO: a
  whitespace-density cut for regions XY-cut cannot separate cleanly.
- Blocks must already be detected upstream (from a PDF extractor or OCR).
  spanwatch orders blocks; it does not find them.