mcp_ocr
Summary: This MCP server provides OCR/extraction tools over local image files, with the same six capabilities offered by three interchangeable backends (Apple Vision, PaddleOCR, PaddleOCR-VL).
Text extraction (
ocr_text_*) — full text plus per-line confidence and bounding boxes.Layout analysis (
ocr_layout_*) — text grouped into blocks with bounding boxes.Table extraction (
ocr_table_*) — structured CSV/JSON output (Vision reportsEUNSUPPORTED; use Paddle/PaddleOCR-VL).Formula extraction (
ocr_formula_paddleocr_vl) — LaTeX from math images.Chart extraction (
ocr_chart_paddleocr_vl) — structured data from charts.Batch OCR (
ocr_batch_*) — pass a list of imagepaths, get consolidated results.Backend choice per task:
vision= fastest, zero-install, macOS GPU/ANE, good CJK/Latin;paddle= cross-platform CPU, 100+ languages, slower (~30–90 s);paddleocr_vl= most accurate 0.9B VLM, 109 languages, tables/formulas/charts, needs the Swift CLI on Apple Silicon.Inputs/options: a filesystem
path(orpathsfor batch), optionallangoverride andmode(baseby default in the schema).Not exposed here: the README's
xiaomibackend tools (document→Markdown, KIE, Xiaomi tables/formulas) are absent from this server schema.
Provides OCR capabilities using Apple Vision framework, enabling text, layout, and table extraction from images on macOS.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp_ocr extract the text from this image"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp_ocr
Modular OCR MCP server with pluggable backends: Apple Vision, PaddleOCR, and Xiaomi-OCR-0 (document parsing, tables, formulas, KIE).
Bundled skill:
ocr-recipes— 12 pluggable OCR workflows (receipt scanner, business card, QR→shopping, document OCR, batch processing, preprocessing guide). Auto-loads in Pi.Requires pi-mcp-bridge to connect to pi.
Quick start
One-line install (pi + pi-mcp-bridge)
Add to ~/.pi/agent/settings.json under mcpBridge.servers:
{
"name": "ocr",
"command": "mcp-ocr",
"args": [],
"setupCommands": [
"uv tool install --force --python 3.11 \"mcp-ocr[all] @ git+https://github.com/timaliev/mcp_ocr.git\""
]
}
setupCommandsruns before each session start. It auto-installs the server with all backends from GitHub. RequiresuvonPATH. Requires pi-mcp-bridge ≥1.0.0.
Restart pi. Done — Vision, Paddle, and Xiaomi-OCR-0 are all available
(Xiaomi-OCR-0 needs the [xiaomi] extra: mcp-ocr[xiaomi]).
Manual install (standalone MCP client)
# All backends (macOS)
uv tool install --python 3.11 "mcp-ocr[all] @ git+https://github.com/timaliev/mcp_ocr.git"
# Vision only (macOS, zero extra deps)
uv tool install --python 3.11 "mcp-ocr[vision] @ git+https://github.com/timaliev/mcp_ocr.git"
# Paddle only (cross-platform, CPU)
uv tool install --python 3.11 "mcp-ocr[paddle] @ git+https://github.com/timaliev/mcp_ocr.git"
# Xiaomi-OCR-0 only (document parsing / tables / formulas / KIE; ~1.6 GB model)
uv tool install --python 3.11 "mcp-ocr[xiaomi] @ git+https://github.com/timaliev/mcp_ocr.git"Then configure your MCP client to run mcp-ocr.
Related MCP server: paddleocr-mcp
Backend dependencies
Group | Backend | Python deps | External deps |
| Apple Vision |
| macOS 10.15+ |
| PaddleOCR |
| Nothing |
| Xiaomi-OCR-0 |
| Model weights (~1.6 GB, downloaded on first use) |
| Everything | vision + paddle + xiaomi | vision + paddle + Xiaomi model |
The Python package mcp-ocr[paddleocr-vl] installs only the subprocess wrapper — not the CLI itself.
Configuration
pi-mcp-bridge with auto-install (recommended)
{
"mcpBridge": {
"servers": [
{
"name": "ocr",
"command": "mcp-ocr",
"args": [],
"setupCommands": [
"uv tool install --force --python 3.11 \"mcp-ocr[all] @ git+https://github.com/timaliev/mcp_ocr.git\""
]
}
]
}
}The setupCommands field (added in pi-mcp-bridge) runs before the server starts. Replace [all] with [vision] or [vision,paddle] to install only specific backends.
Standalone MCP client
{
"mcpServers": {
"ocr": {
"command": "mcp-ocr",
"args": []
}
}
}Backends
Quick comparison
Vision | Paddle | Xiaomi-OCR-0 | |
Speed | ⚡ <100 ms | 🐢 30–90 s | 🐢 2–18 s (after warm-up) |
Accuracy | ★★★★ (Latin/CJK) | ★★★ | ★★★★ (document structure) |
Languages | ~30 locales (CJK, Latin, Cyrillic, Arabic) | 100+ | empirically RU/EN/DE/JA; weak AR/KO |
Platform | macOS 10.15+ | Cross-platform CPU | Apple Silicon (MPS), CUDA, CPU |
RAM usage | ~50 MB | ~2.5 GB | ~2–3 GB (0.8B fp16) |
Install |
|
|
|
First run | Zero | Downloads models (1–2 GB) | Downloads model (~1.6 GB) |
Text | ✅ | ✅ | ✅ |
Layout | ✅ | ✅ | ✅ (blocks; no boxes) |
Table | ❌ (no structure API) | ✅ ( | ✅ OTSL → HTML + CSV |
Formula | ❌ | ❌ | ✅ LaTeX |
Document → Markdown | ❌ | ❌ | ✅ (tables + formulas inline) |
KIE → JSON | ❌ | ❌ | ✅ |
Recommendation: Vision for fast everyday text (and the best Cyrillic on macOS). Xiaomi-OCR-0 when you need document structure — tables, formulas, full-page Markdown, or key-field JSON. Paddle as a cross-platform fallback.
Note on Vision tables: Apple exposes table parsing only through
VNRecognizeDocumentsRequest, which the pinned pyobjc build does not provide.ocr_table_visiontherefore reportsEUNSUPPORTEDand points at Xiaomi/Paddle instead of returning text labeled as a table.
Vision (ocr_text_vision, ocr_layout_vision, ocr_table_vision)
macOS 10.15+, zero install
GPU/ANE accelerated
modedefaults toaccurate— thefastrecognizer is Latin-only and will transliterate Cyrillic/CJK, so only opt into it for quick Latin scansBest for: CJK + major European languages, fast extraction
Paddle (ocr_text_paddle, ocr_layout_paddle, ocr_table_paddle)
Cross-platform CPU
Mature PaddleOCR models,
lang=picks the script family (e.g.ru,ch,japan)Table extraction uses the PaddleOCR 3.x
PPStructureV3pipelineBest for: wide language coverage, when not on macOS
Xiaomi-OCR-0 (ocr_document_xiaomi, ocr_table_xiaomi, ocr_formula_xiaomi, ocr_kie_xiaomi, ocr_text_xiaomi, ocr_layout_xiaomi)
0.8B document-parsing VLM (xiaomi-research/Xiaomi-OCR-0, Apache-2.0), served locally via
transformerson MPS/CUDA/CPU (torchvisionis required by the Qwen3-VL image processor)ocr_document_xiaomi→ Markdown with HTML tables and LaTeX formulasocr_table_xiaomi→headers/rows/csv/html(OTSL parsed by the server)ocr_kie_xiaomi→ JSON; passfieldsto narrow the schemaocr_formula_xiaomi→ LaTeXNo language parameter: it transcribes the document's own language. Measured character error rate 0.00 on printed Russian (including a degraded photo) and English/German/Japanese; Arabic and Korean are unreliable
Model is loaded lazily, so server start stays instant; it is loaded once per server process (~3 GB resident) and every later page reuses it
Input is downscaled to at most
XIAOMI_OCR_MAX_PIXELS(default 4 MP) before inference: every 32×32 pixel block becomes one vision token, so a 300 DPI A4 page (3507×2480) otherwise costs ~8.6k tokens and a single call takes ~40 sOne request at a time: a second concurrent call returns
EBUSYimmediately instead of competing for the single MPS device — retry it in a few secondsIdle unload: the ~3 GB model is released after
XIAOMI_OCR_IDLE_UNLOADseconds without requests (default 300), so an idle agent session does not hold it. The next call reloads it in ~3–8 s.0unloads after every request,-1keeps it resident for the process lifetimeFirst call returns
EWARMINGinstead of blocking: the ~1.75 GB checkpoint downloads in a background thread (progress is reported in the reply). Repeat the call in a minute; once the download is complete the model loads (~20–30 s) and the request runs. This keeps a cold start inside the MCP tool-call timeoutA failed download is reported as
EPROCESSINGwith the underlying error; if the hub is unreachable, pointXIAOMI_OCR_MODELat a local checkpoint directory
Long-running calls (progress notifications)
A dense page can take longer than a client's per-request timeout (a 58-row table
takes ~60 s to decode). The server reports MCP progress every
MCP_OCR_PROGRESS_INTERVAL seconds while a backend call runs; clients that reset
their timeout on progress (Pi, and most progress-aware clients) keep the call alive
and show the elapsed time. Batch tools report per-page progress. Clients that do not
request progress (DSH today) are unaffected — for those, raise the client's
per-request timeout instead (e.g. toolCallTimeoutMs for the DSH MCP client).
Environment Variables
Variable | Default | Description |
|
| Model id or local directory |
| auto ( | Torch device |
|
| Generation budget per request |
|
| Expected checkpoint size, used for EWARMING progress |
|
| Downscale input images above this pixel count (0 disables scaling) |
|
| Seconds without requests before the model is unloaded (0 = after every request, -1 = never) |
|
| Seconds between MCP progress notifications during a long call |
Development
git clone https://github.com/timaliev/mcp_ocr.git
cd mcp_ocr
pip install -e .[all]
pytestMost Xiaomi-OCR-0 tests inject a stub generator, so they run without torch or the
model weights; only the end-to-end test that calls the real model needs [xiaomi].
Available Tools
15 toolsocr_batch_paddleB
OCR multiple images at once. Returns consolidated results.
Backend: paddle. PaddleOCR CPU — cross-platform, mature model. Wider language support than Vision, but slower. Requires PaddleOCR Python package.
Inputs: paths (required), lang, mode.
ocr_batch_visionB
OCR multiple images at once. Returns consolidated results.
Backend: vision. Apple Vision OCR — fast on-device GPU/ANE inference (macOS 10.15+). Best for CJK + major European languages. Zero install on macOS.
Inputs: paths (required), lang, mode.
ocr_batch_xiaomiB
OCR multiple images at once. Returns consolidated results.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: paths (required), lang, mode.
ocr_document_xiaomiB
Parse a document image into Markdown: tables become HTML tables, formulas stay LaTeX, reading order preserved. Returns {'markdown': ...}.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required).
ocr_formula_xiaomiA
Extract mathematical formulas from an image. Returns LaTeX.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required).
ocr_kie_xiaomiA
Extract key information from a document image as a JSON object. Pass 'fields' (list of names) to narrow the output schema.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required), fields.
ocr_layout_paddleB
Extract text with layout analysis. Returns blocks with bounding boxes.
Backend: paddle. PaddleOCR CPU — cross-platform, mature model. Wider language support than Vision, but slower. Requires PaddleOCR Python package.
Inputs: path (required), lang.
ocr_layout_visionB
Extract text with layout analysis. Returns blocks with bounding boxes.
Backend: vision. Apple Vision OCR — fast on-device GPU/ANE inference (macOS 10.15+). Best for CJK + major European languages. Zero install on macOS.
Inputs: path (required), lang, mode.
ocr_layout_xiaomiA
Extract text with layout analysis. Returns blocks with bounding boxes.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required).
ocr_table_paddleB
Extract tables from an image. Returns structured CSV/JSON.
Backend: paddle. PaddleOCR CPU — cross-platform, mature model. Wider language support than Vision, but slower. Requires PaddleOCR Python package.
Inputs: path (required).
ocr_table_visionB
Extract tables from an image. Returns structured CSV/JSON.
Backend: vision. Apple Vision OCR — fast on-device GPU/ANE inference (macOS 10.15+). Best for CJK + major European languages. Zero install on macOS.
Inputs: path (required), lang, mode.
ocr_table_xiaomiB
Extract tables from an image. Returns structured CSV/JSON.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required).
ocr_text_paddleB
Extract text from an image. Returns full text, per-line confidence, and bounding boxes.
Backend: paddle. PaddleOCR CPU — cross-platform, mature model. Wider language support than Vision, but slower. Requires PaddleOCR Python package.
Inputs: path (required), lang.
ocr_text_visionB
Extract text from an image. Returns full text, per-line confidence, and bounding boxes.
Backend: vision. Apple Vision OCR — fast on-device GPU/ANE inference (macOS 10.15+). Best for CJK + major European languages. Zero install on macOS.
Inputs: path (required), lang, mode.
ocr_text_xiaomiB
Extract text from an image. Returns full text, per-line confidence, and bounding boxes.
Backend: xiaomi. Xiaomi-OCR-0 — local 0.8B document-parsing VLM (transformers; Apple Silicon MPS, CUDA or CPU). Best structure fidelity: Markdown output, HTML tables, LaTeX formulas, key-information extraction. Strong on Cyrillic, English, German and Japanese; Arabic/Korean are weak.
Inputs: path (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 15 tools
Names encode both capability and backend (ocr_{capability}_{backend}), so an agent can distinguish 'table' from 'layout' from 'text' tasks and see the backend choice explicitly. The main ambiguity is intentional duplication: ocr_text_vision, ocr_text_paddle and ocr_text_xiaomi do the same job, and choosing between them requires reading descriptions (speed vs language vs installation). Within the xiaomi family, ocr_layout_xiaomi, ocr_document_xiaomi and ocr_kie_xiaomi overlap in structured-output scope, though the descriptions do draw the lines.
A strict, predictable ocr_<capability>_<backend> pattern consistently applied across all 15 tools, with occasional longer capability names (document, formula, kie) that still read naturally. No mixing of camelCase, no stray verb styles, and the backend suffix lets an agent infer an entire tool family from one name.
15 tools is near the top of a comfortable range, but the count is explained by a sensible capability × backend matrix rather than ad-hoc sprawl. It is slightly heavy because each capability is triplicated across three backends, which could have been a single backend parameter; still, each tool corresponds to a real, distinct invocation.
Covers the core OCR surface well: plain text, layout blocks, tables, formulas, full-document Markdown, key-information extraction, and batch processing across three backends. Gaps are minor but notable: no capability/backend discovery tool (an agent cannot check which backends are installed or supported), and features like formula/document/KIE exist only under the xiaomi backend, so workflow coverage is uneven across backends.
Maintenance
Related MCP Connectors
Hosted MCP server: convert PDFs to clean, LLM-ready Markdown with tables, formulas and OCR.
Document-to-Markdown MCP server — convert PDF, Office and HTML into LLM-ready Markdown.
Document conversion MCP server: PDF to Markdown, image OCR, spreadsheet parsing.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Related MCP Servers
- AlicenseNot gradedqualityNot gradedmaintenanceA macOS-based MCP server that enables high-accuracy text extraction from PDF and image files using the OwlOCR app or Apple's Vision Framework. It supports multi-language OCR and provides asynchronous tools for processing documents directly within MCP clients.MIT
- AlicenseCqualityDmaintenanceA local OCR MCP server that extracts text from images using PP-OCRv6 for fast text extraction and VL-1.6 for document structure analysis, with automatic model routing and GPU detection.32MIT
- AlicenseAqualityDmaintenanceMCP server for image recognition, supporting multiple vision backends (Anthropic, Zhipu, Ollama) to describe, answer questions, and analyze images.355 npm1MIT
- FlicenseNot gradedqualityBmaintenanceModular OCR MCP server with pluggable backends (Apple Vision, PaddleOCR, PaddleOCR-VL) for extracting text, tables, formulas, and charts from images.-