VisionCLI
VisionCLI's MCP server lets an AI assistant see your screen, inspect your project's database read-only, drive your Chrome/Brave browser, and start voice mode.
See the screen:
lookscreenshots the front-most window, a display, a named app, a window id, or the window under the cursor;list_windowslists visible windows;select_regionhas the user drag a box;clipboard_imagereturns a clipboard image;device_screenshotcaptures an iOS Simulator or Android device cleanly.Query the database (read-only):
db_schemashows tables, columns and foreign keys (MySQL/Postgres/SQLite from.env);db_queryruns one SELECT/SHOW/DESCRIBE/EXPLAIN/WITH query in a read-only transaction, capped at 50 rows.Control the browser:
browser_tabs,browser_open,browser_switch_tab,browser_navigate(back/forward/reload/close/new),browser_read(page title, URL, selected and readable text),browser_elements(clickable elements with ids),browser_click,browser_type(optionally submitting),browser_scroll.Start voice mode:
start_voiceopens the overlay so the user can hold Option+Space, ask aloud, and get a bubble answer.
Captures clean screenshots from Android devices via adb for visual analysis.
Provides fast Brave browser control via AppleScript, including tab management, navigation, reading pages, clicking, typing, and scrolling.
Reads project .env files to discover database connection settings for read-only MySQL, Postgres, and SQLite queries.
Integrates with Gemini CLI and the Gemini API via API-key auth, enabling voice and typed questions with access to VisionCLI's vision tools.
Captures clean screenshots from iOS Simulator for visual analysis.
Opens file:line references in JetBrains IDEs and detects JetBrains project windows for context.
Integrates with macOS system features for screen capture, microphone/speech input, menu bar presence, automation, and AppleScript-based control.
Supports read-only MySQL schema inspection and SELECT/SHOW/DESCRIBE/EXPLAIN/WITH queries using connection details from the project's .env.
Maps PhpStorm project windows to their local project folders so questions are answered about the correct project.
Supports read-only SQLite schema inspection and SELECT/SHOW/DESCRIBE/EXPLAIN/WITH queries using connection details from the project's .env.
Interacts with terminal apps such as Warp to type questions into running Claude/Gemini CLI sessions and receive answers.
Uses Xcode for iOS Simulator screenshots and can open file:line references in Xcode.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@VisionCLI what is this error on my screen?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
"What is this file for?" Your assistant looks at the window under your mouse, finds the real file in your project and answers with
file:linereferences.Hold ⌥ and ask out loud (⌥⌘ to include a screenshot). Visi, a little robot, peeks in from the edge of the screen while it listens and thinks; the answer appears in the top-right corner and is read aloud.
Claude or Gemini, your choice. Uses your own
claudeoragysign-in (no API keys) and the session already running for that project, so you can type and talk in the same conversation. Pick the model and effort in VisionCLI menu → Settings… → Assistant.Uses your computer. "Open Xcode and run the app", "fill this form", "switch the dropdown to Production": it clicks, types, presses shortcuts and opens apps, one checked step at a time, reading each app's real buttons and fields through Accessibility. You approve each action (or allow all), passwords and System Settings always ask, a task panel shows every step with Pause/Stop, and Esc stops it instantly.
Meeting Notes (optional). In a meeting, it transcribes your colleagues (your computer's sound) and you (the microphone) on your computer and shows the conversation in a side panel. Each colleague's turn gets a one-line summary. When you're asked something, three short replies appear once they've finished talking. Questions about the code are answered from your project. Each meeting is saved with its notes, a summary at the end and a recording you can play back. Nothing is sent for you.
Notices crashes. When your app (Mac or Simulator) crashes, VisionCLI offers to explain it, and the next question already knows which crash report to read.
Points and draws. Answers point at things on your real screen (boxes, circles, arrows, numbered steps) and can include Mermaid diagrams rendered in the bubble.
Fast browser control. "Open the Stripe tab", "click sign in", "pause the video" run in under a second (via TypeSafe Jev), with a visible AI cursor. Forms, dropdowns, toggles and framed portals work through the page itself, without opening stray tabs.
Multi-step browser tasks. "Find a stay in Lisbon with a pool and free cancellation and open it": a Jev agent works through the page in front one step at a time (type, click, pick, scroll), about 0.3–0.5 s per step, with each step shown in the bubble. Needs Chrome's Allow remote debugging (chrome://inspect/#remote-debugging) and a click on Allow when it first connects.
Knows your data. Read-only database questions ("how are payments and orders related?") using your project's
.env.
Install
Requirements
macOS 13 or later (Apple Silicon or Intel), Linux (x86_64 or arm64; X11 or Wayland: see Linux), or Windows 10/11 (x64 or Arm, preview: see Windows)
At least one AI CLI, installed and signed in: Claude Code (
claude), Antigravity CLI (agy, Gemini on a Google account), or Gemini CLI (gemini)Node.js 18 or later
1. Download the app
Download
VisionCLI-<version>-macos.dmgfrom the latest release.Open it and drag VisionCLI into Applications.
The app isn't notarized by Apple yet, so the first time:
Right-click VisionCLI → Open → Open (on macOS 15 and later: System Settings → Privacy & Security → Open Anyway), or
run
xattr -dr com.apple.quarantine /Applications/VisionCLI.appin Terminal.
With Homebrew (available once published):
brew install --cask thethirdsourcers/tap/visioncliEvery release file can be checked against the workflow that built it: gh attestation verify <file> -R thethirdsourcers/visioncli. All install options are on visioncli.thethirdsourcers.com; docs/DISTRIBUTION.md covers each channel.
Updates
VisionCLI checks GitHub for new releases at launch and every 6 hours. When one is out, the VisionCLI menu shows Install Update: it downloads the release, checks it's signed with the same certificate, replaces the app and restarts, keeping your permissions and settings. Use VisionCLI menu → Settings… → General → Check Now to check now, or turn automatic checks off there.
2. First launch
The VisionCLI icon appears in the menu bar and a welcome screen offers to set up the CLIs it finds. That registers VisionCLI's tools for every project: claude mcp add visioncli --scope user … for Claude Code, the global MCP config for Antigravity CLI, and gemini mcp add -s user --trust visioncli … for Gemini CLI. You can also do it later from VisionCLI menu → Settings… → Integrations → Connect.
macOS will ask for three permissions. VisionCLI needs all of them:
Permission | Why |
Microphone and Speech Recognition | to hear your question while you hold ⌥ |
Screen & System Audio Recording for VisionCLI | to see the window you're asking about in voice mode. After allowing it, quit and reopen VisionCLI. |
Screen & System Audio Recording for your terminal app (iTerm, Terminal, Warp…) | when you type to Claude Code, the screenshot is taken by the terminal running |
Automation (iTerm, Chrome/Brave) | asked the first time it types into your terminal or controls the browser |
If something doesn't work, check System Settings → Privacy & Security, and VisionCLI menu → Settings… → Advanced → Open Log.
3. Optional extras
Add | Get | How |
Parakeet | Fastest local transcription (~0.3s on the Neural Engine) that spells your project's class names right. Apple Silicon | VisionCLI menu → Settings… → Voice → Speech recognition → Parakeet (downloads ~600 MB on first use) |
Wispr Flow | Use your own Wispr Flow account: hold ⌥ and speak as usual | Have Flow running with its push-to-talk key on fn, then pick Wispr Flow in Speech Recognition |
Whisper | More accurate transcription of names and jargon (~0.7s, local) | VisionCLI menu → Settings… → Voice → Speech recognition → Whisper Download… (installs |
Kokoro | Natural voices for spoken answers (local) | VisionCLI menu → Settings… → Voice → Natural voices Download… (installs |
TypeSafe Jev | Instant browser commands and multi-step browser tasks | add your key via VisionCLI menu → Settings… → Integrations → Edit API Keys…, or set |
Browser control | Click, scroll and read pages | in Chrome or Brave: View → Developer → Allow JavaScript from Apple Events |
Models are saved in ~/Library/Application Support/visioncli/models/. Whisper and Kokoro models you already have from other tools (e.g. in ~/.cache/whisper) are picked up automatically. Without them, VisionCLI uses Apple's on-device speech recognition and the system voice.
Related MCP server: PeepIt MCP
Linux
Download
VisionCLI-<version>-linux-x86_64.AppImage(or-arm64), the.debfor Debian/Ubuntu or the.rpmfor Fedora/openSUSE, from the latest release. Or install from a repository, so updates come with the system's (available once published):# Debian / Ubuntu curl -fsSL https://visioncli.thethirdsourcers.com/public.key | sudo gpg --dearmor -o /usr/share/keyrings/visioncli.gpg echo "deb [signed-by=/usr/share/keyrings/visioncli.gpg] https://visioncli.thethirdsourcers.com stable main" | sudo tee /etc/apt/sources.list.d/visioncli.list sudo apt update && sudo apt install visioncli-linux # Fedora / RHEL / openSUSE sudo curl -fsSL -o /etc/yum.repos.d/visioncli.repo https://visioncli.thethirdsourcers.com/visioncli.repo sudo dnf install visioncli-linux # Flathub / Snap Store flatpak install flathub com.thethirdsourcers.VisionCLI sudo snap install visioncli --classicInstall the desktop tools VisionCLI uses (the script detects apt/dnf/pacman/zypper and your desktop, and asks before installing):
sh app/scripts/install-deps.shfrom the repo, or see docs/LINUX.md.Start VisionCLI. The setup wizard checks each piece and offers fixes:
Hold Alt to talk reads the keyboard directly, so join the
inputgroup once (sudo usermod -aG input $USER, then log out and in). On X11 it also works without it.GNOME on Wayland: VisionCLI installs a small GNOME Shell extension so it can see windows and the pointer (log out and in once). KDE, Sway and Hyprland need nothing extra.
Computer control on Wayland asks once through the desktop's Remote Desktop permission.
Terminal mode types into
clauderunning in tmux, kitty or WezTerm, or into any terminal window directly.
Keys: ⌥ is Alt, ⌘ is Super. Everything else matches the macOS app.
Windows
Preview. Download VisionCLI-<version>-windows-x64.exe (or -arm64) from the latest release and run it: it installs for your user only, no admin rights. The installer isn't code-signed yet, so SmartScreen asks once: More info → Run anyway. With winget (available once published): winget install TheThirdSourcers.VisionCLI; a Microsoft Store listing is in preparation.
Keys: ⌥ is Alt, ⌘ is the Win key (hold Alt to ask, Alt + Win with a screenshot, Alt + Shift + Space to drag a region). Terminal mode types into claude in Windows Terminal, WezTerm or any console window, including Claude Code inside WSL. Nothing else to install: keyboard, screen, UI Automation and the microphone work out of the box; the setup wizard downloads the speech models.
Gemini (Antigravity CLI or Gemini CLI)
VisionCLI works with Gemini as well as Claude, using your own sign-in, the same way it uses your claude login:
Use when | |
Antigravity CLI ( | You sign in with a personal Google account. Google now serves personal accounts through Antigravity. |
Gemini CLI ( | Your Gemini CLI sign-in still works (paid Code Assist / work account), or you add an API key. |
Set up: VisionCLI menu → Settings… → Integrations → Connect next to Gemini (Antigravity) or Gemini CLI (or
visioncli install, which registers every CLI it finds).Antigravity: adds visioncli to
~/.gemini/config/mcp_config.jsonand allows its read-only/navigation tools in~/.gemini/antigravity-cli/settings.json.Gemini CLI:
gemini mcp add -s user --trust visioncli ….
Typing: in
agyorgemini, the visioncli tools (look, browser, database, device screenshots…) are available in every workspace.Voice: choose VisionCLI menu → Settings… → Assistant → Gemini (Antigravity).
Terminal mode types your question into the
agy/geminirunning for the project (or opens one) and shows the answer in the terminal and the bubble.Bubble Only mode runs it headless (
agy -p … --output-format stream-json, orgemini -p … -o stream-json) and continues the same conversation each time.
Pointing at a terminal that runs
claude,agyorgeminiuses that CLI, whatever the Assistant setting.
Notes:
In Bubble Only mode the CLI can't ask for approval, so it only reads (code, screen, browser, database). Use terminal mode for edits and commands.
Headless
agytakes ~15s to start per question; terminal mode keeps it running and answers faster.Gemini CLI on a free personal Google sign-in now fails with "…no longer supported for Gemini Code Assist for individuals". Use Antigravity instead, or add
"gemini": {"apiKey": "..."}in VisionCLI menu → Settings… → Integrations → Edit API Keys….
Use it
Voice (anywhere)
Keys | What happens |
Hold ⌥, speak, let go | Ask as text, no screenshot (cheap). The assistant knows which window you're over and takes a look itself if it needs to. Tap to start, tap again to send |
Hold ⌥ ⌘ | Ask with a screenshot of the window under the mouse |
Hold ⌥ ⇧ Space | Ask with a screenshot of just an area: drag a box around exactly what you mean, or tap ⌥ ⇧ Space again for the whole window |
⌥ P | Pin the answer: scroll, select text, click |
⌥ O | Open the first |
⌥ ↩ / esc | Approve or deny an edit or command (background mode) |
esc | Dismiss, or stop the answer |
To attach a screenshot to every question instead, turn on VisionCLI menu → Settings… → General → Screenshot every question.
Which project? By default VisionCLI follows the window under your mouse. PhpStorm myshop – payments maps to ~/PhpstormProjects/myshop, and a terminal running claude maps to its folder. With several projects open, pick one in VisionCLI menu → Project. The menu bar shows the current project.
Which session? In Terminal mode (the default), voice questions are typed into the claude (or agy/gemini, see Gemini) already running for that project in iTerm, or a new one is opened. You can type there too; it's one conversation, and answers show in both the terminal and the bubble. Bubble Only mode uses a hidden, leaner session instead. Switch in VisionCLI menu → Settings… → Assistant → Answers appear.
In Claude Code (typing)
After Set Up Claude Code, just talk about what's on screen in any project:
"what is this file for?", "why is this view misaligned in the simulator?"
"let me select an area", "read the page I have open and summarize it"
"how are the orders and payments tables related?"
Slash commands: /mcp__visioncli__see <question>, /mcp__visioncli__point <question>.
Tool | |
| Screenshot the window behind the terminal, the one under the mouse, a named app, or a display; reports the pointer position |
| Pick a window, drag a box, or use the clipboard image |
| Clean iOS Simulator (needs Xcode) or Android ( |
| Tables, columns, foreign keys; single read-only SELECT/SHOW/DESCRIBE/EXPLAIN/WITH queries in a read-only transaction (MySQL, Postgres, SQLite from |
| Fast Chrome/Brave control via AppleScript (dropdowns, toggles with on/off state, custom React/Radix widgets) |
| Fills a whole form in one call: text fields, dropdowns (native, custom or searchable) and checkboxes |
| Hands a multi-step task on the open page to the Jev agent (about 0.4 s a step) |
| Use any app: Accessibility element list with ids, real mouse/keyboard, by element id or screenshot pixels. Performed by VisionCLI.app (needs Accessibility permission), approved in the bubble |
| Start the voice overlay |
Menu
Project (Auto, or pick from running claude/agy/gemini sessions, open editors and recent projects) · Claude Usage · History · New Conversation · Copy Last Answer · Show Terminal · Settings… (⌘,).
Settings… opens one window, laid out like System Settings; changes apply right away:
General: start at login, check for updates (now or automatically), Setup Wizard, screenshot every question.
Assistant: Claude Code or Gemini (Antigravity), where answers appear (terminal or bubble only), model, effort, fast model for simple questions.
Voice: read answers aloud, speed, voice (with a sample), natural voice download, speech recognition (Parakeet, Wispr Flow, Whisper, Apple), vocabulary, microphone and speaker.
Computer Control: ask before each action, allow all, or off; Accessibility; AI cursor and instant commands in the browser; watch for app crashes.
Meeting Notes: on/off, summaries (colleagues, you), suggested replies, code answers, recording, past meetings, and hiding VisionCLI from screen sharing.
Integrations: connect VisionCLI's tools to Claude Code, Antigravity and Gemini CLI; API keys.
Advanced: Fix Permissions, log, crash log, home folder.
API keys
VisionCLI menu → Settings… → Integrations → Edit API Keys… opens ~/Library/Application Support/visioncli/integrations.json (readable only by you):
{
"typesafe": { "apiKey": "..." },
"gemini": { "apiKey": "..." }
}typesafe: instant browser commands with Jev. Without it, every request goes to the assistant.gemini(optional): makes VisionCLI's Gemini runs use this API key instead of yourgeminisign-in.
TYPESAFE_API_KEY in your shell is copied here by visioncli voice if the file has none.
Privacy
Screenshots and questions go to the assistant you picked (Claude Code or Antigravity), through your own login.
With Jev enabled, quick commands send your spoken words, open tab titles and URLs, and visible link/button text to api.typesafe.ai. Browser tasks also send the visible text, controls and field values of the page in front (never password fields). Turn it off in Settings.
Computer control runs locally: the MCP server talks to VisionCLI.app over 127.0.0.1 with a per-launch secret token (in a file only you can read). Nothing is clicked or typed unless an assistant asks and you allow it (or chose Allow All Actions).
Whisper and Kokoro run locally. History is stored in
~/Library/Application Support/visioncli/.
Build from source
git clone https://github.com/thethirdsourcers/visioncli.git && cd visioncli
npm install # builds the MCP server (dist/)
node dist/index.js install # register with Claude Code, Antigravity CLI and Gemini CLI (whichever are installed)
npm run build # build VisionCLI.app and install it into /Applications
node dist/index.js voice # start voice mode (under launchd: restarts after a crash)npm run setup-signing(optional, once) creates a local code-signing identity so macOS keeps permissions across rebuilds.npm run releasebuilds the downloadable universal.dmgand.zipintorelease/. Pushing av*tag does the same on GitHub Actions and attaches them to a release.
Layout: src/ is the MCP server and CLI (TypeScript). overlay/Sources/ is the menu-bar app (Swift). scripts/ has the build, release and signing scripts.
Troubleshooting
Problem | Fix |
"VisionCLI can't be opened" | Right-click → Open, or |
Claude sees only the wallpaper | Voice: enable Screen Recording for VisionCLI. Typing in Claude Code: enable it for your terminal app. Then quit and reopen that app. |
Tools missing in Claude Code / Gemini CLI | VisionCLI menu → Settings… → Integrations → Connect / Reconnect (again after moving the app or switching Node versions), then restart |
Gemini CLI: "no longer supported for Code Assist for individuals" | Google moved personal accounts to Antigravity: choose VisionCLI menu → Settings… → Assistant → Gemini (Antigravity) (or add a Gemini API key) |
Gemini answers "tried a step that needs your approval" | Bubble Only mode can't approve commands/edits; switch VisionCLI menu → Settings… → Assistant → Answers appear to the terminal |
Answers only in the terminal, or none at all | VisionCLI menu → Settings… → Advanced → Open Log; check |
Microphone errors | Pick a specific mic in VisionCLI menu → Settings… → Voice → Microphone |
Sound bars move but no text appears | The pill now says why. Usually: allow Speech Recognition for VisionCLI (Privacy & Security), or turn on Keyboard → Dictation. Installing Whisper also transcribes without Apple's recognizer. |
Asked for permissions again after updating, though Settings shows VisionCLI as allowed | The switch belongs to an older signature. VisionCLI 0.5.5+ clears it automatically on first launch; otherwise use VisionCLI menu → Settings… → Advanced → Fix Permissions… (or remove VisionCLI with − and add it again). Happens once when moving to the permanently signed releases; later updates keep permissions. |
It crashed | It restarts automatically; details are in VisionCLI menu → Settings… → Advanced → Open Crash Log |
Contributing
Contributions are welcome: bug reports, fixes, platform support and docs. See CONTRIBUTING.md for how the code is organised and how to build and test it, and look for issues labelled good first issue to get started. Security problems: SECURITY.md.
License
MIT © Third Sourcers
Available Tools
31 toolsannounceTell the user out loudA
Show a short update in VisionCLI's bubble and say it out loud, e.g. when background work you promised to report on has finished ("The v0.7.6 release is live."). One or two short spoken sentences, plain text, no markdown.
Inputs: text (required).
browser_clickClick an elementB
Click an element by id from browser_elements (or by its visible text).
Inputs: target (required).
browser_dragDrag and dropA
Drag an element onto another (ids from browser_elements, which lists draggables, or visible text): sorting lists, boards, upload drop zones, sliders built from divs. Returns the target's text afterwards.
Inputs: to (required), from (required).
browser_elementsList clickable elementsARead-only
Visible links, buttons, fields, dropdowns (with options) and toggles (with on/off state) on the active page, with ids (e1, e2, …) for browser_click / browser_type / browser_select.
Inputs: no named parameters.
browser_fillFill in a formA
Fill several fields of a form in one call (ids from browser_elements): text fields get the text, dropdowns (native, custom or searchable) pick the option by its text, checkboxes/radios/switches are set on (true) or off (false). Much faster than one call per field. Each field reports its result; submit with browser_click afterwards.
Inputs: fields (required).
browser_hoverHover over an elementA
Hover over an element (id from browser_elements) to show what appears on hover: menus, captions, tooltips. Then call browser_elements to see and click what appeared.
Inputs: target (required).
browser_openOpen a URLA
Open a URL in Chrome/Brave. Always use this (never ⌘L/⌘T + typing) to go to a page: it switches to a tab already showing it, otherwise reuses your own working tab, and only opens one new tab the first time. new_tab: false loads it in the user's current tab.
Inputs: url (required), new_tab.
browser_readRead the current pageARead-only
Title, URL, selected text and readable text of the active tab (capped). Faster and more exact than a screenshot.
Inputs: max_chars.
browser_scrollScroll the pageC
Scroll the active page.
Inputs: where (required).
browser_selectChoose a dropdown optionA
Pick an option in a dropdown (native or custom combobox/menu) by its visible text. Use this instead of browser_click for dropdowns. Toggles/switches/checkboxes: use browser_click (browser_elements shows their on/off state).
Inputs: option (required), target (required).
browser_switch_tabSwitch tabB
Bring a tab (id from browser_tabs) to the front.
Inputs: tab_id (required).
browser_tabsList browser tabsARead-only
Open tabs in the front-most Brave/Chrome (ids like t1.3 = window 1, tab 3), marking the active one.
Inputs: no named parameters.
browser_taskDo a task on the page (fast)A
Hand a multi-step task on the web page in front of the user's Chrome/Brave to VisionCLI's Jev agent: fill in a form, search this site and open a result, set filters, pick options, add to a cart. It clicks, types and selects in a few seconds (about 0.4 s a step). Put every value to type in the goal: it only types text that's in the goal. Returns each step and where it ended; check the result with browser_read if needed. Open the right page first (browser_open).
Inputs: goal (required).
browser_typeType into a fieldB
Set the value of a form field (id from browser_elements), optionally submitting the form.
Inputs: text (required), target (required), submit.
clickClickA
Click on the screen: by element id from ui_elements (preferred), or at x/y pixels in the last screenshot (from look, or image N of the voice question). Supports right-click and double-click.
Inputs: x, y, count, image, +4 more.
clipboard_imageRead image from clipboardBRead-only
Return the image currently on the user's clipboard (e.g. a screenshot taken with Cmd+Ctrl+Shift+4).
Inputs: no named parameters.
db_queryRun a read-only SQL queryARead-only
Run one SELECT/SHOW/DESCRIBE/EXPLAIN/WITH query against the project's database inside a read-only transaction. Results are capped at 50 rows; use LIMIT.
Inputs: sql (required).
db_schemaDatabase schemaARead-only
Tables, columns and foreign keys of the project's database (connection read from the project's .env: Laravel DB_* or DATABASE_URL; MySQL, Postgres, SQLite). Pass a table to narrow it. Read-only.
Inputs: table.
device_screenshotScreenshot a simulator or deviceARead-only
Full-resolution screenshot of the booted iOS Simulator (platform ios) or a connected Android device/emulator (platform android), without window chrome. Use it to explain or debug app UI, then find the view in code.
Inputs: platform.
dragDragB
Drag with the left mouse button from (x1,y1) to (x2,y2): move files, select text, resize, reorder.
Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +3 more.
list_windowsList visible windowsARead-only
List on-screen app windows (front-most first) with ids, app names and titles. Use before look when unsure which window the user means.
Inputs: no named parameters.
lookLook at the screenARead-only
Take a screenshot of what the user is looking at and return it as an image. By default captures the window under the mouse pointer (or, if that is the terminal, the front-most other window). Use target "screen" for the whole display the mouse is on. Optionally pick a window by app name/title or window id, or a display number.
Inputs: app, target, display, window_id, +1 more.
open_appOpen or switch to an appA
Open an app, or bring it to the front if it's running, e.g. "Xcode", "Slack", "Simulator", "Android Studio", "Figma".
Inputs: name (required), look_after.
open_urlOpen a URLB
Open a link or file URL with its default app (browser, Figma, Slack deep links, vscode://, etc.).
Inputs: url (required), look_after.
press_keysPress keysA
Press a key or shortcut (not for browser navigation: use browser_open/browser_navigate), e.g. "enter", "cmd+s", "cmd+shift+p", "tab", "esc", "down". Several separated by spaces run in order: "cmd+a delete".
Inputs: keys (required), app, look_after.
scrollScrollB
Scroll at x/y (default: where the mouse is). dy negative = down, positive = up (lines); dx for sideways.
Inputs: x, y, dx, dy, +3 more.
select_regionLet the user select a regionARead-only
Show the native crosshair so the user can drag a rectangle around the exact part of the screen they mean (press Space to pick a window instead, Esc to cancel). Blocks until they finish. Tell the user to make the selection.
Inputs: no named parameters.
start_voiceStart voice modeA
Launch the voice overlay for this project: the user holds Option (Option+Command with a screenshot), asks a question out loud, and the answer appears in a bubble above their mouse pointer.
Inputs: no named parameters.
type_textType textA
Type text with the keyboard into the focused field, or into element (clicked first). replace: true sets the field's whole value instead of typing. Use press_keys for Enter, Tab, shortcuts.
Inputs: text (required), app, element, replace, +1 more.
ui_elementsList UI elementsARead-only
List the buttons, fields, menus, links, tabs and text of the front app's focused window (or a named app) via Accessibility, each with an id and its center in screen points. Use the id with click/type_text: it's far more reliable than pixel guessing.
Inputs: app.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 31 tools
Descriptions do distinguish the browser_* family (web pages) from the unprefixed desktop tools, which helps, but there are many near-duplicate pairs: browser_click/click, browser_type/type_text, browser_scroll/scroll, browser_drag/drag, and browser_elements/ui_elements. browser_task also overlaps nearly every other browser tool, so an agent must read carefully to avoid misselection.
All names use snake_case, but conventions are mixed: browser_* tools are consistent, while desktop tools swap between noun-only (look), verb (click, scroll, drag), and verb_noun (type_text, press_keys, open_app, open_url). The browser_ prefix scoping is a plus, yet the unprefixed half isn't predictable.
31 tools exceeds the comfortable 15-tool band and sits in the 'heavy' range, but the server genuinely spans browser automation, native app control, database inspection, and device screenshots, so most tools earn their place. A few (e.g. the browser_task meta-tool plus individual browser actions) feel redundant rather than strictly necessary.
The surface covers a broad lifecycle: page discovery, click/type/select/drag/scroll, tab and navigation control, plus desktop input and app launching, with db read and device screenshots. Gaps are minor (db is read-only, no explicit browser_close/tab management beyond switch), and agents can work around them.
Maintenance
Related MCP Connectors
Use your Mac, Windows or Linux computer from ChatGPT, Claude or Codex: files, commands, documents.
Mac & Windows: let ChatGPT, Claude & Cursor use your email, calendar, iMessage, Teams, files.
Your AI apps as one team: hand work to Codex or Claude Code on your Mac, with a shared logbook.
Connect ChatGPT, Claude, Cursor or VS Code to your Windows PC: files, terminal, browser, apps.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables Claude to see and interact with any macOS application using natural language commands. Perfect for testing Mac applications, UI automation, and app development with AI assistance.16 PyPI34MIT
- AlicenseAqualityDmaintenanceEnables AI agents to capture and analyze screenshots of macOS applications, windows, or the entire screen using local (Ollama) or cloud-based AI vision models, with non-intrusive, fast screen capture via Apple's ScreenCaptureKit.323 npm2MIT
- AlicenseNot gradedqualityDmaintenanceScreenshot and diagram tool for AI agents. Capture and annotate screenshots to show Claude what you mean — or let the agent render Mermaid diagrams and open them for visual review. Approve, annotate, or request changes with text feedback. Built-in review mode with structured responses. CLI and MCP server for Claude Code, Cursor, Windsurf, Cline. macOS, open source, free.5 npm352MIT
- AlicenseNot gradedqualityCmaintenanceProvides AI coding assistants with privacy-first, synchronized screen context including cursor position, active window metadata, and visual verification to aid in building and debugging graphical interfaces.MIT