mcp-vision
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-visionopen Notepad and type 'Hello, world!'"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-vision
Local-first screen perception + actuation over the Model Context Protocol. A host (Claude Desktop, Cursor, or agent.py) calls tools; the server looks at the display, numbers the controls, and clicks only after a safety governor (and, for restricted actions, a transparent HUD confirm).
Quick Guide · Agent Instructions
Quickstart
# uv
uv pip install -e .
# pipx (stdio MCP server on PATH)
pipx install .
mcp-vision doctor # display / accessibility / backends
mcp-vision install # write Claude Desktop + Cursor MCP config
mcp-vision serve # stdio JSON-RPC (logs on stderr only)Host config (also written by mcp-vision install):
{
"mcpServers": {
"mcp-vision": {
"command": "mcp-vision",
"args": ["serve"]
}
}
}Paths: ~/Library/Application Support/Claude/claude_desktop_config.json and ~/.cursor/mcp.json.
HUD overlay (optional): pip install -e ".[hud]" then restricted clicks draw a red box — Space confirm / Esc abort / timeout auto-pauses.
Related MCP server: puppet-ai
Architecture
flowchart TB
Host["Claude / Cursor / agent.py"] -->|stdio JSON-RPC| Server["mcp-vision serve"]
Server --> Inspect["inspect_screen"]
Inspect --> Capture["mss capture + JPEG downscale"]
Capture --> Parser["SoM boxes + optional OCR"]
Parser --> Governor{"governor"}
Governor -->|SAFE_READ / ROUTINE_WRITE| Actuate["click / type / hotkey"]
Governor -->|RESTRICTED| HUD["transparent HUD confirm"]
HUD -->|Space| Actuate
HUD -->|Esc / timeout| Abort["ActionResult.ok=false"]Tool | Returns |
|
|
|
|
|
|
|
|
Restricted labels (password, delete, buy, checkout, terminal, cmd+q, …) never auto-run. No confirmer → hard abort.
Mac agent (same repo)
The MCP server is the vision/actuation layer. agent.py is the local Plan → Act → Reflect loop (AppleScript + Playwright DOM, eval gates, traces, judge, skills). Local Ollama by default.
python agent.py --as general "make a note called Ideas with a haiku about the sea"
python agent.py --as web-researcher "summarize the top story on news.ycombinator.com"
python agent.py --as general --model claude "..."
python agent.py --tui --as general "..."See QUICK_GUIDE.md for specialists, eval gates, and the observability flywheel.
graph LR
A[Task] --> B[qwen3:8b picks a tool]
B --> C{Tool}
C -->|Notes / Reminders / Calendar| D[AppleScript]
C -->|web_*| E[Playwright AX + SoM fallback]
D --> V{eval gate}
E --> V
V --> F[trace.jsonl + OTLP runs/id/trace.json]
F --> BTests
Hermetic unit tests use synthetic screens — no physical display, no stdout pollution of MCP.
PYTHONPATH=src pytest tests/test_capture.py tests/test_parser.py tests/test_governor.py tests/test_mcp_tools.py tests/test_config_sync.py
python tests/run.py # existing module self-checks
python bench/runner.py # dry agent loop
python tests/e2e_web.py # headless Playwright workflowsThis server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Let ChatGPT, Claude & Cursor use your Mac: email, calendar, iMessage, Teams, files. Local, free.
Autonomous cloud agent tasks: real browser + your tools, structured evidence-backed results.
Build agents to automate any background task. Works with your ChatGPT/Claude subscription.
Turns any agent into a full agentic application — branded, interactive screens generated at runtime.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to see and control your desktop with tools for screenshots, clicks, typing, and more, all locally on macOS and Windows.64MIT
- AlicenseNot gradedqualityDmaintenanceFull computer control for AI agents on macOS — see, click, type, scroll via MCP. Works with any app, all processing on-device with PII protection.MIT
- AlicenseNot gradedqualityBmaintenanceAutomate your real Chrome browser locally with AI, supporting vision, human-like input, code execution, macros, and watchdogs.15MIT
- AlicenseNot gradedqualityAmaintenanceLets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.2MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/hussainn7/mcp-vision'
If you have feedback or need assistance with the MCP directory API, please join our Discord server