OCR MCP

An HTTP (Streamable) MCP server written in Node.js that receives a file and a structured JSON object with empty values, extracts the file's content using local OCR / text extraction (no cloud, no API keys), and fills in the JSON values.

Supported file types:

How field filling works

Filling happens in two passes:

1. Heuristic parser (always on, local). Matches each JSON key to a label in the document text, e.g. a key invoiceNumber matches lines like Invoice Number: INV-2026 or invoice_number - INV-2026. Nested objects are supported.

2. LLM fallback (optional). For any field the heuristic could not resolve, a configured LLM is asked to fill the remaining values from the extracted text. It only fills still-empty fields and never overrides values already found. Enable it via .env (see below); leave it disabled to stay fully local.

Fields that remain empty after both passes are returned in an unfilled list.

Related MCP server: pymupdf4llm-mcp

Requirements

• Node.js 18+ (developed on v24)

Install

npm install
cp .env.example .env   # then edit .env if you want the LLM fallback

Configuration (.env)

Provider defaults for LLM_BASE_URL:

• openai → https://api.openai.com/v1

• anthropic → https://api.anthropic.com

• ollama → http://localhost:11434/v1

openai and ollama use the OpenAI-compatible Chat Completions API; anthropic uses the Messages API. Any OpenAI-compatible endpoint (Azure OpenAI, LM Studio, vLLM, etc.) works by pointing LLM_PROVIDER=openai at a custom LLM_BASE_URL.

Run

npm start
# OCR MCP server (Streamable HTTP) listening on http://localhost:3000/mcp
# LLM fallback enabled: openai / gpt-4o-mini   (or "disabled")

Health check: GET /health.

MCP tool

fill_json_from_file

* Provide at least one of mimeType or filename.

Returns JSON (as text + structuredContent):

{
  "data": { "name": "Acme Corp", "invoiceNumber": "INV-2026", "missingField": "" },
  "unfilled": ["missingField"],
  "extractedTextLength": 41,
  "usedLlm": false
}

usedLlm indicates whether the LLM fallback was invoked. If a fallback call fails, an llmError field is included and the heuristic result is still returned.

Connecting from an MCP client

Point a Streamable-HTTP-capable MCP client at http://localhost:3000/mcp. Example VS Code mcp.json:

{
  "servers": {
    "ocr": {
      "type": "http",
      "url": "http://localhost:3000/mcp"
    }
  }
}

Example (raw HTTP)

# 1. initialize -> read the mcp-session-id response header
# 2. POST notifications/initialized with that session id
# 3. POST tools/call with the base64 file + schema
# See the curl handshake used in development; Accept must include
# "application/json, text/event-stream".