Invoice Vision MCP

An MCP server that lets AI agents read invoice images. It uses computer vision (OpenCV + Tesseract OCR) to turn a scan or phone photo into structured data, checks it against Saudi VAT rules, and stores it in SQL so an LLM can answer questions like "How much did we spend on IT services last month?"

Why

Finance teams still type invoices into spreadsheets by hand. LLMs are good at answering questions about the data, but they can't open your files or your database safely. MCP is the standard way to give a model controlled access to tools and data. This server is that bridge: any MCP client (Claude Desktop, Cursor, VS Code, or the Business Analyst Agent built with LangGraph) can ingest invoices and analyse spend.

Related MCP server: Doc Agent

Architecture

flowchart LR
    A[Invoice image<br/>scan / phone photo] --> B[OpenCV preprocessing<br/>grayscale · denoise · deskew · threshold]
    B --> C[Tesseract OCR<br/>text + confidence]
    C --> D[Parser<br/>regex → Pydantic]
    D --> E[Validator<br/>VAT 15% · totals · VAT no.]
    E --> F[(SQLite)]
    F --> G[MCP server<br/>tools · resources · prompts]
    G <--> H[MCP clients<br/>Claude Desktop · Cursor · LangGraph agent]

MCP interface

Results

Measured with python scripts/evaluate.py on 12 generated invoices. One in three is a simulated phone photo (rotated ±4°, blurred, noisy), and 2 contain deliberate errors.

Honest limitation: the data is synthetic and uses one clean layout. Real invoices vary widely, so a rule-based parser will miss fields on unseen layouts. The roadmap below covers how I would handle that.

Security

An MCP server runs with real permissions, so it's built defensively:

• Path sandbox. Files are resolved inside INVOICE_INBOX only, so ../../etc/passwd is rejected (tested).

• Read-only SQL. The SQL tool uses a keyword filter, allows one statement only, and opens a mode=ro SQLite connection (tested).

• Typed inputs. Every tool argument is validated through its type-hinted signature.

• No secrets. The server needs no API keys, and everything runs locally.

Quick start

# 1. System dependency
sudo apt install tesseract-ocr        # macOS: brew install tesseract

# 2. Install
git clone https://github.com/Abdullah-Alroumi/invoice-vision-mcp
cd invoice-vision-mcp
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# 3. Demo data and tests
python -m invoice_vision.samples --out samples
pytest -q
python scripts/evaluate.py

# 4. Inspect the server in the official MCP Inspector
npx @modelcontextprotocol/inspector invoice-vision-mcp

Use it from Claude Desktop

Add this to claude_desktop_config.json:

{
  "mcpServers": {
    "invoice-vision": {
      "command": "/absolute/path/to/.venv/bin/invoice-vision-mcp",
      "env": {
        "INVOICE_INBOX": "/absolute/path/to/samples",
        "INVOICE_DB": "/absolute/path/to/data/invoices.db"
      }
    }
  }
}

Then ask: "Ingest the new invoices and show me spend by category. Which ones failed validation?"

Run over HTTP (Docker)

docker build -t invoice-vision-mcp .
docker run -p 8000:8000 -v $PWD/samples:/app/samples invoice-vision-mcp
# MCP endpoint: http://localhost:8000/mcp

Configuration

Project structure

src/invoice_vision/
├── preprocess.py   # OpenCV pipeline (deskew via projection profile)
├── ocr.py          # Tesseract wrapper with confidence score
├── parser.py       # OCR text → Invoice, plus VAT business rules
├── models.py       # Pydantic schemas
├── pipeline.py     # image → validated ExtractionResult
├── db.py           # SQLite repository + read-only query guard
├── samples.py      # synthetic invoice generator (demo + tests)
└── server.py       # MCP tools / resources / prompts
scripts/            # accuracy evaluation
tests/              # parser, vision, DB, MCP-client and security tests

Roadmap

• Vision-language model fallback. When OCR confidence is low or validation fails, send the image to a VLM for layout-free extraction, and keep the rule-based path as the cheap default.

• Arabic invoices (OCR_LANG=eng+ara) and the ZATCA e-invoice QR code (TLV decoding).

• Layout detection (YOLO / LayoutParser) to find the totals and line-item regions before OCR.

• PostgreSQL and auth for multi-user HTTP deployments.

License

MIT. All sample companies and numbers are fictional.