[1] User drops a document
[2] doc-extract MCP  ← this repo. Reads the WHOLE document, returns JSON
[3] DB node          → insert into NeonDB          (separate node)
[4] Agent node       → chats over the NeonDB content (separate node)
[5] Or: the team acts on the JSON directly, with no DB at all

# ONE WORLD TRADING L.L.C.

## Key fields
| Field | Value |
|---|---|
| supplier_code | S00066 |
| currency | AED |
| ageing_date | 2025-07-11 |

### Line items
| document_no | bp_reference_no | due_date | amount | running_balance |
|---|---|---|---|---|
| 131365 | SI/08781/CN/00007 | 2025-07-07 | -43160.25 | -43160.25 |
...

## All document fields (as printed)
| Posting Date | From To 11.07.25 |
| Sales Employee | No Sales Employee |
...

{
  "schema_version": "2.0",
  "status": "ok",                     // ok | needs_review | parsed_without_profile
                                      // | profile_mismatch | no_text_layer | error
  "profile": "sap_b1_supplier_statement",
  "profile_confidence": 1.0,
  "document": { "file_name": "...", "checksum": "sha256:...",
                "pages": 1, "pages_parsed": [0] },
  "metadata": { "supplier_name": "ONE WORLD TRADING L.L.C.",
                "supplier_code": "S00066", "currency": "AED",
                "ageing_date": "2025-07-11" },
  "line_items": [
    { "line_no": 1, "document_type": "PU", "document_no": "131365",
      "bp_reference_no": "SI/08781/CN/00007",
      "posting_date": "2025-05-31", "due_date": "2025-07-07",
      "amount": -43160.25, "running_balance": -43160.25,
      "_source": {                    // only when include_coordinates=true
        "page": 0,
        "cells": { "BP Ref. No.": { "page": 0, "wrapped": true,
                                    "bbox": [123.7, 278.67, 163.79, 295.02] } }
      } }
  ],
  "summary":    { "buckets": { "Balance Due": -69966.75 } },
  "validation": { "ok": true, "checks": [ ... ] },
  "diagnostics": { "rows": 5, "rows_with_wrapped_cells": 1,
                   "column_fill_rate": { ... }, "page_geometry": [ ... ],
                   "warnings": [] }
}

PASS  clean data validates
PASS  misread amount on line 2      -> running_balance_chain, sum_equals_last
PASS  rows out of order             -> running_balance_chain
PASS  duplicated row                -> running_balance_chain
PASS  mangled reference number      -> field_matches:bp_reference_no   <-- ONLY this
PASS  unclaimed words on page       -> word_coverage, no_unassigned_words
PASS  summary disagrees             -> summary_equals_last
PASS  missing required field        -> required_fields

bash scripts/check_repo.sh                      # is the clone complete?
bash scripts/run_tests.sh                       # all 6 suites, no server
npx @modelcontextprotocol/inspector python -m src.server   # see it as a client
python scripts/smoke_test.py <url> <token> doc.pdf         # verify a deployment

pip install -r requirements.txt
export DOC_EXTRACT_TOKEN=$(openssl rand -hex 32)
MCP_TRANSPORT=http python -m src.server     # http://0.0.0.0:8000/mcp

python tests/test_samples.py                # parser regression
python tests/test_detection.py              # validation fires
python tests/e2e_http.py                    # real MCP client over HTTP

docker build -t doc-extract .
docker run -p 8000:8000 -e DOC_EXTRACT_TOKEN=$TOKEN doc-extract
curl localhost:8000/health

{
  "status": "profile_mismatch",
  "header_missing": "Post. Date",
  "header_actual": ["Document","BP Ref. No.","Posting Date","Due Date",
                    "Details","Amount","Balance"],
  "next_step": "Update its `columns` to the printed header, then
                POST /admin/reload-profiles."
}

status: needs_review
refs  : ['SI/2000', 'SI/2001', 'SI/2100']      <-- NOT corrupted
FAILED: no_orphan_lines  {'text': 'STRAY-FRAGMENT', 'reason': 'before_first_row'}

id: acme_invoice
detect:
  require: ["Tax Invoice"]
  text_contains: ["Tax Invoice", "Invoice No."]
table:
  columns: ["Line", "Item Code", "Description", "Qty", "Amount"]
  anchor_column: "Line"
  anchor_pattern: '^\d+$'
  stop_pattern: '^Subtotal\b'
  join_with: ""          # "" for codes/refs, " " for prose
fields:
  - {name: item_code, source: "Item Code", type: text}
  - {name: amount,    source: "Amount",    type: decimal}
validation:
  - {type: required_fields, fields: [item_code, amount]}

[1] Trigger: user drops a document / Outlook attachment
        ↓
[2] Agent node: extract_document(source=<url>, file_name=...)
        ↓
    switch on status:
      ok                     -> [3] insert -> [4] chat agent
      needs_review           -> human approval -> insert / reject
      parsed_without_profile -> [3] insert anyway (content is complete)
                                 + alert: new vendor format seen
      no_text_layer          -> OCR queue
      error                  -> retry, then alert
        ↓
[3] DB node: run neon_schema.sql once, then upsert on document.checksum
        ↓
[4] Agent node with NeonDB access:
      "what does this say?"  -> SELECT markdown FROM documents WHERE ...
      "how much is past due?" -> SELECT SUM(amount) FROM v_document_lines ...