MCP-Vision

An open-source contextual action layer for your computer. Point at what you're working on, invoke MCP-Vision, and let your preferred model Ask, Guide, or Act—with verification and evidence for meaningful actions.

Your model. Your computer. Evidence for every action.

Needs Python 3.12+ (macOS /usr/bin/python3 is often 3.9 and will fail).

Install (one liner)

curl -fsSL https://raw.githubusercontent.com/hussainn7/mcp-vision/main/scripts/install.sh | bash

That script installs uv if needed, fetches Python 3.12, and puts mcp-vision on ~/.local/bin.

Manual

# if you don't have 3.12 yet:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv python install 3.12

uv tool install "git+https://github.com/hussainn7/mcp-vision.git" --python 3.12
export PATH="$HOME/.local/bin:$PATH"
mcp-vision setup

Do not use stock pip3 on macOS if it reports 3.9.

Devs (clone)

cd mcp-vision
python3.12 -m venv .venv   # or: uv venv --python 3.12
source .venv/bin/activate
pip install -e .
mcp-vision setup

Then open Chrome and refresh MCP in Cursor / Claude.

Related MCP server: open-computer-use

Contextual invocation on macOS

mcp-vision doctor
mcp-vision ui

Press Option-Space anywhere to open the small MCP-Vision popup beside the cursor. It collects the foreground application, window, selection, and focused accessibility element when macOS makes those fields reliably available. A simple Ask is answered in place; the inferred Ask · Guide · Act labels do not create a second automation engine.

For Chrome, open chrome://extensions, enable Developer mode, choose Load unpacked, and select this repository's chrome_relay folder. Right-click a page and choose Ask MCP-Vision. The action sends a bounded selection/element/nearby DOM context to the same local runtime and opens the native popup. Keep mcp-vision ui running while using the action.

Useful checks:

mcp-vision status
mcp-vision install --host cursor

The UI is optional. Existing mcp-vision serve, MCP host configuration, and CLI workflows remain independent.

Runtime boundary

The core owns context, orchestration, trust decisions, verification, receipts, and UX. Browser and desktop control sit behind an execution-backend protocol; the existing native/CDP/isolated runtimes are the defaults and are not duplicated.

Ask the bot

mcp-vision ask "flights to SFO from ATL Sept 28 to Oct 2"
mcp-vision ask "what's on my gmail"
mcp-vision ask "mechanical keyboard on ebay under 100"

Uses your existing Chrome (native, no automation banner). On success a second model pass (Ollama by default) cleans the raw page into a short answer. Use --model none for a local heuristic only.

Connect an agent host

mcp-vision setup --host cursor
mcp-vision setup --host claude-desktop
mcp-vision setup --host antigravity

Or print config: mcp-vision config --allow-browser-writes

Claude Code:

claude mcp add --transport stdio mcp-vision -- mcp-vision serve --browser live --driver native --allow-browser-writes

Then ask the host in plain English: list my tabs and find flights to SF…

How live Chrome works

serve --browser live (default --driver native) drives the Chrome you already have open — cookies, extensions, tabs. No remote-debugging toggle. No “controlled by automated software” banner.

CAPTCHA? You get a notification — solve it in Chrome, click I solved it. Buy / send / book still need Allow once.

Quick checks

mcp-vision connect
mcp-vision demo
mcp-vision probe --live
mcp-vision studio
mcp-vision status

License

MIT