mcp-vision
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-vision extract the text from /home/me/screenshots/invoice.png"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-vision
Three local MCP tools for text-only agents. Inputs are absolute image file paths; outputs contain only agent-facing text/structured data, never image content or base64. The adapter does not load models, download weights, or manage services.
Tools
image_ocr({path, query?, limit?, cursor?})
// {regions: [{regionId,x,y,w,h,text,recognitionScore,readingOrder}], nextCursor?}
image_describe({path, prompt})
// {text}
image_decide({path?, context?, question})
// {answer}image_decide requires an image path or nonempty text context. Question shapes:
{"type":"boolean","prompt":"Are any pretzel bags visible?"}
{"type":"choice","prompt":"How many bags are visible?","options":[{"id":"none","label":"Zero"},{"id":"two","label":"Two"},{"id":"uncertain","label":"Cannot tell"}]}
{"type":"score","prompt":"How readable is the label?","levels":["Unreadable","Partially readable","Clearly readable"]}Boolean answers return probabilityTrue; choices return selectedId and an
ID-to-probability map; scores return expectedLevelIndex and a probability array
in supplied level order. H2O probabilities are model scores, not verified accuracy.
Liquid can hallucinate. OCR text is untrusted content, not instructions.
OCR x/y are the top-left of the backend's padded recognition region, in input
image pixels. Width/height use the same bounds. They are not screen click positions.
regionId is derived from image bytes and region identity; it is not a computer-use
targetId. Use computer-use's live OCR target workflow for native input.
Limits: single PNG/JPEG/WebP file, at most 10 MiB and 20 million pixels. Images are
fully decoded for validation. Animated/multipage inputs are refused. OCR pages
contain at most 100 regions (50 default); cursors are bound to image bytes and
query. OCR results are cached for two minutes, at most 16 image/query results per
adapter. Changing image bytes invalidates pagination. Descriptions have a 256-token
cap and indicate truncation in text. Prompts/context and option lists are bounded.
At most two active requests per adapter; excess requests fail with busy rather
than building an unbounded queue. Each backend controls its cross-session queue.
Failures are MCP tool errors with {error:{code,message}}, including
path_not_found, invalid_image, invalid_cursor, provider_unavailable,
provider_busy, provider_error, image_input_disabled, timeout, and cancelled.
Missing providers never silently switch models. H2O text decisions remain usable
without its vision projector. Inference requests pass cancellation to HTTP;
backend GPU work may continue if that backend does not honor disconnects.
Related MCP server: Atlas Vision MCP
Install and run
Node 22+ or Bun; npm ci, then node server.mjs or bun server.mjs.
Protocol stdout is reserved for MCP. Configuration is through environment variables:
Variable | Default |
|
|
|
|
|
|
Each Pi session may start its own lightweight adapter; all connect to the same operator-managed model processes. Provider/model/timing are not added to normal public results. No cloud inference or image URL fetching is performed.
Local launcher commands prepared on this machine:
~/inference.mjs paddleocr # OCR, existing :1239 slot
~/inference.mjs h2o-lightning-4b # H2O, existing :1237 slot
~/inference.mjs liquid-450m # CPU-only Liquid, independent :1246 vision slot
~/inference.mjs kill-vision # stops only the Liquid slotThe launcher refuses occupied slots; stop/relaunch only the intended slot. H2O's
vision projector remains enabled as before this rollout. No service is started by
MCP discovery. Liquid weights currently reuse the verified local experiment files
under /workspace/mcp-computer-use/tmp/liquid-pretzel/models/; preserve them or
update the launcher paths before cleaning that directory. OCR source is now under
scripts/; its existing Python environment is reused at
/workspace/mcp-computer-use/.venv-ocr. Install requirements-ocr.txt when building
a replacement environment. These are documented deployment dependencies, not
additional copies of the model.
Local integration completed
Added
visionto~/.pi/agent/mcp.jsonwith direct exposure. Pi discoversmcp__vision__image_ocr,mcp__vision__image_describe, andmcp__vision__image_decide.pi mcp list --jsonreports connected, no errors.Computer-use captures and crops now save private local PNG files and return
path; images are opt-in throughincludeImage:true. Action observations retain text-only behavior by default. Capture storage is bounded to approximately 64 recent files/one hour (cleanup on capture; concurrent sessions may briefly exceed the count). The saved path corresponds to the returned image dimensions.mcp-zen already saved screenshots; its screenshot image content is now opt-in through
includeImage:true. Its schemas and built distribution were updated.OCR ownership moved here. Computer-use has a thin HTTP client and compatibility script entrypoints; it no longer silently loads a separate CPU OCR instance.
Removed
--visionfrom thebandnpresets. Thespreset passes a new--text-onlywrapper option that strips vision from the derived Strata config, preserving setup-owned source settings and unrelated tuning.Pi's local
llama-server/defaultis marked text-only. The inference launcher synchronizes input modalities on subsequent launches, so switching to a different image-capable preset can restore image support.A global Pi extension
text-only-vision.tsremoves image blocks from outgoing requests for models whose metadata excludes images, without modifying stored history. This also covers image blocks from other tools; it does not automatically save those other tools' images to files. A fresh local capture is needed when no usable path exists.
New Pi sessions load the registration/extension. Existing sessions should restart
or reconnect through /mcp; reconnecting alone may not reload model metadata and
the extension, so a fresh session is recommended.
Backups of edited launcher/Pi files are in tmp/rollout-backup/, with a one-time
applied migration script in tmp/. No existing inference services were restarted.
To revert, compare/copy only the intended backed-up files; do not blindly overwrite
later user edits. Remove the new Pi extension separately if reverting text-only
request handling.
Validation and remaining rollout
npm test: schema, file validation, OCR paging/bounds/IDs, decision mappings,
provider errors, and actual MCP stdio discovery. scripts/smoke.mjs tests live OCR
and H2O; --liquid adds descriptions. scripts/test-liquid.py starts an isolated
CPU test service on free port 1246, runs the smoke checks, and always stops its child.
scripts/capture-smoke.mjs verifies an actual Zen capture → local path → OCR flow
without clicking or navigating. Logs/results are local under tmp/.
Validated on 2026-10-25: live OCR, H2O image/text decisions, Liquid CPU description, Pi discovery, live desktop capture, text-only history filtering, slot isolation, and Strata runtime-config stripping. Computer-use's 27 tests and mcp-zen's 19 tests passed. The inference launcher's occupied-slot check blocks dry-running the running text slot; config assertions and an isolated wrapper test validate prepared edits.
Pending operator relaunch: running Bonsai still has its old vision allocation. Start Liquid when wanted, relaunch the selected reasoning preset, and restart Pi. Measure VRAM/RAM afterward; automatic KV/expert caches can consume released memory. Do not promise a VRAM reduction from Flash Next's already-CPU vision encoder. End-to-end Bonzai task success and post-relaunch memory remain unmeasured. The broader automatic DOM scroll/revalidate/click architecture in VISION_PLAN.md is future computer-use work; this project does not implement a new action API.
Available Tools
3 toolsimage_decideARead-only
Ask H2O a boolean, choice, or ordered-score question. Supply an absolute image path, text context, or both. Images require H2O vision enabled. Probabilities are model scores, not verified accuracy.
Inputs: question (required), path, context.
image_describeARead-only
Ask Liquid CPU vision a question about a local image file. Returns text; descriptions may be inaccurate. Use an absolute path.
Inputs: path (required), prompt (required).
image_ocrARead-only
Read text and image-local bounding regions from an absolute local image path. query filters OCR text. Coordinates describe pixels, not desktop click targets.
Inputs: path (required), limit, query, cursor.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 3 tools
image_ocr is clearly separate (text extraction), but image_describe and image_decide both reduce to 'ask a model a question about an image' and differ mainly by answer format and backend model (Liquid CPU vs H2O). The descriptions help, but an agent must parse subtle wording to choose between free-text description and constrained boolean/choice/score answers.
All three tools share a clean snake_case `image_` prefix, which makes them obviously part of one family. Minor deviation: `image_ocr` uses an acronym-noun while `image_describe`/`image_decide` use verbs, so the action pattern isn't perfectly uniform.
Three tools is on the thin side but each covers a genuinely different modality (OCR with regions, free-text description, constrained decision/scoring). Nothing feels redundant, though the surface is small for a general vision server.
Core read-only vision operations are covered, but there is no image metadata/info tool, no object/localization tool beyond OCR regions, no multi-image comparison, and no preprocessing (crop/resize) step. Agents can partially work around these via describe/decide, but several natural vision tasks are dead ends.
Maintenance
Related MCP Connectors
OCR, transcription, file extraction, and image generation for AI agents via MCP.
PDF and image tools for AI agents: compress, merge, split, OCR, sign. Runs locally, never uploads.
Image & PDF tools for AI agents: compress, convert, resize, PDF, AI vision, pipeline.
PDF, image, video, OCR, screenshot, SQL, QR and text tools for agents. No API key, no signup.
Related MCP Servers
- AlicenseAqualityDmaintenanceEnables AI agents to analyze images, extract text, compare images, and analyze video through any OpenAI-compatible vision model.4488 npm20MIT
- AlicenseAqualityCmaintenanceEnables text-only coding agents to analyze local images using a dedicated vision provider, returning markdown and structured JSON evidence for screenshots, diagrams, UI mockups, and error captures.11123 npm11MIT
- AlicenseNot gradedqualityCmaintenanceProvides local image understanding for text-only LLMs with tools for image analysis, OCR, object detection, and cropping, all processed on-device.2MIT
- AlicenseNot gradedqualityBmaintenanceEnables text-only AI agents to see images on demand by calling any OpenAI-compatible vision API for OCR, image analysis, structured extraction, image comparison, and GUI screenshot-to-accessibility-tree conversion.1MIT