# Invoice Vision MCP

**An MCP server that lets AI agents read invoice images.** It uses computer vision
(OpenCV + Tesseract OCR) to turn a scan or phone photo into structured data, checks it
against Saudi VAT rules, and stores it in SQL so an LLM can answer questions like
*"How much did we spend on IT services last month?"*

![Preprocessing pipeline](docs/preprocessing.png)

| | |
|---|---|
| **Protocol** | Model Context Protocol (MCP): tools, resources and prompts over stdio or Streamable HTTP |
| **Computer vision** | OpenCV denoising, projection-profile deskew, adaptive thresholding → Tesseract OCR |
| **Extraction** | Rule-based parser → Pydantic models (vendor, invoice no., date, VAT no., line items, totals) |
| **Validation** | 15% VAT check, total = subtotal + VAT, line items sum to the subtotal, 15-digit VAT number |
| **Storage** | SQLite with duplicate protection and a read-only SQL tool for analysts and agents |
| **Quality** | 21 automated tests, including a real MCP client session and security tests |

---

## Why

Finance teams still type invoices into spreadsheets by hand. LLMs are good at answering
questions about the data, but they can't open your files or your database safely.
**MCP** is the standard way to give a model controlled access to tools and data. This
server is that bridge: any MCP client (Claude Desktop, Cursor, VS Code, or the
[Business Analyst Agent](https://github.com/Abdullah-Alroumi/business-analyst-agent) built
with LangGraph) can ingest invoices and analyse spend.

## Architecture

```mermaid
flowchart LR
    A[Invoice image<br/>scan / phone photo] --> B[OpenCV preprocessing<br/>grayscale · denoise · deskew · threshold]
    B --> C[Tesseract OCR<br/>text + confidence]
    C --> D[Parser<br/>regex → Pydantic]
    D --> E[Validator<br/>VAT 15% · totals · VAT no.]
    E --> F[(SQLite)]
    F --> G[MCP server<br/>tools · resources · prompts]
    G <--> H[MCP clients<br/>Claude Desktop · Cursor · LangGraph agent]
```

## MCP interface

| Type | Name | What it does |
|---|---|---|
| Tool | `list_inbox` | New vs already-ingested images in the inbox |
| Tool | `extract_invoice` | Run CV + OCR and return structured fields (does not save) |
| Tool | `ingest_invoice` | Extract, validate and save; duplicates are rejected |
| Tool | `list_invoices` | Filter by vendor, category, date range, or invalid only |
| Tool | `get_invoice` | One invoice with its line items and validation issues |
| Tool | `spending_report` | Count, subtotal, VAT and total grouped by vendor, category or month |
| Tool | `query_invoices_sql` | Read-only `SELECT` for questions the other tools don't cover |
| Resource | `invoices://schema` | DB schema, so the model writes correct SQL |
| Resource | `invoices://summary` | Headline numbers |
| Prompt | `monthly_spend_report` | Reusable template for a monthly finance report (EN + AR) |

## Results

Measured with `python scripts/evaluate.py` on 12 generated invoices. One in three is a
simulated phone photo (rotated ±4°, blurred, noisy), and 2 contain deliberate errors.

| Metric | Result |
|---|---|
| Field accuracy (7 header fields) | **100%** (84/84) |
| Line-item count match | **100%** |
| Error detection (bad invoices flagged, good ones passed) | **100%** |
| Mean OCR confidence | ~94% |
| Time per invoice (CPU) | ~1.5 s |

> **Honest limitation:** the data is synthetic and uses one clean layout. Real invoices vary
> widely, so a rule-based parser will miss fields on unseen layouts. The roadmap below covers
> how I would handle that.

## Security

An MCP server runs with real permissions, so it's built defensively:

- **Path sandbox.** Files are resolved inside `INVOICE_INBOX` only, so `../../etc/passwd` is rejected (tested).
- **Read-only SQL.** The SQL tool uses a keyword filter, allows one statement only, *and* opens a `mode=ro` SQLite connection (tested).
- **Typed inputs.** Every tool argument is validated through its type-hinted signature.
- **No secrets.** The server needs no API keys, and everything runs locally.

## Quick start

```bash
# 1. System dependency
sudo apt install tesseract-ocr        # macOS: brew install tesseract

# 2. Install
git clone https://github.com/Abdullah-Alroumi/invoice-vision-mcp
cd invoice-vision-mcp
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# 3. Demo data and tests
python -m invoice_vision.samples --out samples
pytest -q
python scripts/evaluate.py

# 4. Inspect the server in the official MCP Inspector
npx @modelcontextprotocol/inspector invoice-vision-mcp
```

### Use it from Claude Desktop

Add this to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "invoice-vision": {
      "command": "/absolute/path/to/.venv/bin/invoice-vision-mcp",
      "env": {
        "INVOICE_INBOX": "/absolute/path/to/samples",
        "INVOICE_DB": "/absolute/path/to/data/invoices.db"
      }
    }
  }
}
```

Then ask: *"Ingest the new invoices and show me spend by category. Which ones failed validation?"*

### Run over HTTP (Docker)

```bash
docker build -t invoice-vision-mcp .
docker run -p 8000:8000 -v $PWD/samples:/app/samples invoice-vision-mcp
# MCP endpoint: http://localhost:8000/mcp
```

## Configuration

| Variable | Default | Meaning |
|---|---|---|
| `INVOICE_INBOX` | `./samples` | Only folder the server may read images from |
| `INVOICE_DB` | `./data/invoices.db` | SQLite path |
| `OCR_LANG` | `eng` | Tesseract languages, e.g. `eng+ara` |
| `MCP_TRANSPORT` | `stdio` | `stdio` or `streamable-http` |
| `MCP_HOST` / `MCP_PORT` | `127.0.0.1` / `8000` | Bind address for HTTP transport |

## Project structure

```
src/invoice_vision/
├── preprocess.py   # OpenCV pipeline (deskew via projection profile)
├── ocr.py          # Tesseract wrapper with confidence score
├── parser.py       # OCR text → Invoice, plus VAT business rules
├── models.py       # Pydantic schemas
├── pipeline.py     # image → validated ExtractionResult
├── db.py           # SQLite repository + read-only query guard
├── samples.py      # synthetic invoice generator (demo + tests)
└── server.py       # MCP tools / resources / prompts
scripts/            # accuracy evaluation
tests/              # parser, vision, DB, MCP-client and security tests
```

## Roadmap

- **Vision-language model fallback.** When OCR confidence is low or validation fails, send the image to a VLM for layout-free extraction, and keep the rule-based path as the cheap default.
- **Arabic invoices** (`OCR_LANG=eng+ara`) and the ZATCA e-invoice QR code (TLV decoding).
- **Layout detection** (YOLO / LayoutParser) to find the totals and line-item regions before OCR.
- **PostgreSQL** and auth for multi-user HTTP deployments.

## License

MIT. All sample companies and numbers are fictional.