Skip to main content
Glama

  • "What is this file for?" Your assistant looks at the window under your mouse, finds the real file in your project and answers with file:line references.

  • Hold ⌥ and ask out loud (⌥⌘ to include a screenshot). Visi, a little robot, peeks in from the edge of the screen while it listens and thinks; the answer appears in the top-right corner and is read aloud.

  • Claude or Gemini, your choice. Uses your own claude or agy sign-in (no API keys) and the session already running for that project, so you can type and talk in the same conversation. Pick the model and effort in VisionCLI menu → Settings… → Assistant.

  • Uses your computer. "Open Xcode and run the app", "fill this form", "switch the dropdown to Production": it clicks, types, presses shortcuts and opens apps, one checked step at a time, reading each app's real buttons and fields through Accessibility. You approve each action (or allow all), passwords and System Settings always ask, a task panel shows every step with Pause/Stop, and Esc stops it instantly.

  • Meeting Notes (optional). In a meeting, it transcribes your colleagues (your computer's sound) and you (the microphone) on your computer and shows the conversation in a side panel. Each colleague's turn gets a one-line summary. When you're asked something, three short replies appear once they've finished talking. Questions about the code are answered from your project. Each meeting is saved with its notes, a summary at the end and a recording you can play back. Nothing is sent for you.

  • Notices crashes. When your app (Mac or Simulator) crashes, VisionCLI offers to explain it, and the next question already knows which crash report to read.

  • Points and draws. Answers point at things on your real screen (boxes, circles, arrows, numbered steps) and can include Mermaid diagrams rendered in the bubble.

  • Fast browser control. "Open the Stripe tab", "click sign in", "pause the video" run in under a second (via TypeSafe Jev), with a visible AI cursor. Forms, dropdowns, toggles and framed portals work through the page itself, without opening stray tabs.

  • Multi-step browser tasks. "Find a stay in Lisbon with a pool and free cancellation and open it": a Jev agent works through the page in front one step at a time (type, click, pick, scroll), about 0.3–0.5 s per step, with each step shown in the bubble. Needs Chrome's Allow remote debugging (chrome://inspect/#remote-debugging) and a click on Allow when it first connects.

  • Knows your data. Read-only database questions ("how are payments and orders related?") using your project's .env.

Install

Requirements

  • macOS 13 or later (Apple Silicon or Intel), Linux (x86_64 or arm64; X11 or Wayland: see Linux), or Windows 10/11 (x64 or Arm, preview: see Windows)

  • At least one AI CLI, installed and signed in: Claude Code (claude), Antigravity CLI (agy, Gemini on a Google account), or Gemini CLI (gemini)

  • Node.js 18 or later

1. Download the app

  1. Download VisionCLI-<version>-macos.dmg from the latest release.

  2. Open it and drag VisionCLI into Applications.

  3. The app isn't notarized by Apple yet, so the first time:

    • Right-click VisionCLI → Open → Open (on macOS 15 and later: System Settings → Privacy & Security → Open Anyway), or

    • run xattr -dr com.apple.quarantine /Applications/VisionCLI.app in Terminal.

With Homebrew (available once published):

brew install --cask thethirdsourcers/tap/visioncli

Every release file can be checked against the workflow that built it: gh attestation verify <file> -R thethirdsourcers/visioncli. All install options are on visioncli.thethirdsourcers.com; docs/DISTRIBUTION.md covers each channel.

Updates

VisionCLI checks GitHub for new releases at launch and every 6 hours. When one is out, the VisionCLI menu shows Install Update: it downloads the release, checks it's signed with the same certificate, replaces the app and restarts, keeping your permissions and settings. Use VisionCLI menu → Settings… → General → Check Now to check now, or turn automatic checks off there.

2. First launch

The VisionCLI icon appears in the menu bar and a welcome screen offers to set up the CLIs it finds. That registers VisionCLI's tools for every project: claude mcp add visioncli --scope user … for Claude Code, the global MCP config for Antigravity CLI, and gemini mcp add -s user --trust visioncli … for Gemini CLI. You can also do it later from VisionCLI menu → Settings… → Integrations → Connect.

macOS will ask for three permissions. VisionCLI needs all of them:

Permission

Why

Microphone and Speech Recognition

to hear your question while you hold ⌥

Screen & System Audio Recording for VisionCLI

to see the window you're asking about in voice mode. After allowing it, quit and reopen VisionCLI.

Screen & System Audio Recording for your terminal app (iTerm, Terminal, Warp…)

when you type to Claude Code, the screenshot is taken by the terminal running claude. Without it, Claude only sees the wallpaper. Restart the terminal after allowing it.

Automation (iTerm, Chrome/Brave)

asked the first time it types into your terminal or controls the browser

If something doesn't work, check System Settings → Privacy & Security, and VisionCLI menu → Settings… → Advanced → Open Log.

3. Optional extras

Add

Get

How

Parakeet

Fastest local transcription (~0.3s on the Neural Engine) that spells your project's class names right. Apple Silicon

VisionCLI menu → Settings… → Voice → Speech recognition → Parakeet (downloads ~600 MB on first use)

Wispr Flow

Use your own Wispr Flow account: hold ⌥ and speak as usual

Have Flow running with its push-to-talk key on fn, then pick Wispr Flow in Speech Recognition

Whisper

More accurate transcription of names and jargon (~0.7s, local)

VisionCLI menu → Settings… → Voice → Speech recognition → Whisper Download… (installs whisper-cpp via Homebrew, then downloads the ~490 MB model)

Kokoro

Natural voices for spoken answers (local)

VisionCLI menu → Settings… → Voice → Natural voices Download… (installs uv if needed, then downloads ~350 MB)

TypeSafe Jev

Instant browser commands and multi-step browser tasks

add your key via VisionCLI menu → Settings… → Integrations → Edit API Keys…, or set TYPESAFE_API_KEY

Browser control

Click, scroll and read pages

in Chrome or Brave: View → Developer → Allow JavaScript from Apple Events

Models are saved in ~/Library/Application Support/visioncli/models/. Whisper and Kokoro models you already have from other tools (e.g. in ~/.cache/whisper) are picked up automatically. Without them, VisionCLI uses Apple's on-device speech recognition and the system voice.

Related MCP server: PeepIt MCP

Linux

  1. Download VisionCLI-<version>-linux-x86_64.AppImage (or -arm64), the .deb for Debian/Ubuntu or the .rpm for Fedora/openSUSE, from the latest release. Or install from a repository, so updates come with the system's (available once published):

    # Debian / Ubuntu
    curl -fsSL https://visioncli.thethirdsourcers.com/public.key | sudo gpg --dearmor -o /usr/share/keyrings/visioncli.gpg
    echo "deb [signed-by=/usr/share/keyrings/visioncli.gpg] https://visioncli.thethirdsourcers.com stable main" | sudo tee /etc/apt/sources.list.d/visioncli.list
    sudo apt update && sudo apt install visioncli-linux
    
    # Fedora / RHEL / openSUSE
    sudo curl -fsSL -o /etc/yum.repos.d/visioncli.repo https://visioncli.thethirdsourcers.com/visioncli.repo
    sudo dnf install visioncli-linux
    
    # Flathub / Snap Store
    flatpak install flathub com.thethirdsourcers.VisionCLI
    sudo snap install visioncli --classic
  2. Install the desktop tools VisionCLI uses (the script detects apt/dnf/pacman/zypper and your desktop, and asks before installing): sh app/scripts/install-deps.sh from the repo, or see docs/LINUX.md.

  3. Start VisionCLI. The setup wizard checks each piece and offers fixes:

    • Hold Alt to talk reads the keyboard directly, so join the input group once (sudo usermod -aG input $USER, then log out and in). On X11 it also works without it.

    • GNOME on Wayland: VisionCLI installs a small GNOME Shell extension so it can see windows and the pointer (log out and in once). KDE, Sway and Hyprland need nothing extra.

    • Computer control on Wayland asks once through the desktop's Remote Desktop permission.

    • Terminal mode types into claude running in tmux, kitty or WezTerm, or into any terminal window directly.

Keys: ⌥ is Alt, ⌘ is Super. Everything else matches the macOS app.

Windows

Preview. Download VisionCLI-<version>-windows-x64.exe (or -arm64) from the latest release and run it: it installs for your user only, no admin rights. The installer isn't code-signed yet, so SmartScreen asks once: More info → Run anyway. With winget (available once published): winget install TheThirdSourcers.VisionCLI; a Microsoft Store listing is in preparation.

Keys: ⌥ is Alt, ⌘ is the Win key (hold Alt to ask, Alt + Win with a screenshot, Alt + Shift + Space to drag a region). Terminal mode types into claude in Windows Terminal, WezTerm or any console window, including Claude Code inside WSL. Nothing else to install: keyboard, screen, UI Automation and the microphone work out of the box; the setup wizard downloads the speech models.

Gemini (Antigravity CLI or Gemini CLI)

VisionCLI works with Gemini as well as Claude, using your own sign-in, the same way it uses your claude login:

Use when

Antigravity CLI (agy)

You sign in with a personal Google account. Google now serves personal accounts through Antigravity.

Gemini CLI (gemini)

Your Gemini CLI sign-in still works (paid Code Assist / work account), or you add an API key.

  • Set up: VisionCLI menu → Settings… → Integrations → Connect next to Gemini (Antigravity) or Gemini CLI (or visioncli install, which registers every CLI it finds).

    • Antigravity: adds visioncli to ~/.gemini/config/mcp_config.json and allows its read-only/navigation tools in ~/.gemini/antigravity-cli/settings.json.

    • Gemini CLI: gemini mcp add -s user --trust visioncli ….

  • Typing: in agy or gemini, the visioncli tools (look, browser, database, device screenshots…) are available in every workspace.

  • Voice: choose VisionCLI menu → Settings… → Assistant → Gemini (Antigravity).

    • Terminal mode types your question into the agy/gemini running for the project (or opens one) and shows the answer in the terminal and the bubble.

    • Bubble Only mode runs it headless (agy -p … --output-format stream-json, or gemini -p … -o stream-json) and continues the same conversation each time.

  • Pointing at a terminal that runs claude, agy or gemini uses that CLI, whatever the Assistant setting.

Notes:

  • In Bubble Only mode the CLI can't ask for approval, so it only reads (code, screen, browser, database). Use terminal mode for edits and commands.

  • Headless agy takes ~15s to start per question; terminal mode keeps it running and answers faster.

  • Gemini CLI on a free personal Google sign-in now fails with "…no longer supported for Gemini Code Assist for individuals". Use Antigravity instead, or add "gemini": {"apiKey": "..."} in VisionCLI menu → Settings… → Integrations → Edit API Keys….

Use it

Voice (anywhere)

Keys

What happens

Hold ⌥, speak, let go

Ask as text, no screenshot (cheap). The assistant knows which window you're over and takes a look itself if it needs to. Tap to start, tap again to send

Hold ⌥ ⌘

Ask with a screenshot of the window under the mouse

Hold ⌥ ⇧ Space

Ask with a screenshot of just an area: drag a box around exactly what you mean, or tap ⌥ ⇧ Space again for the whole window

⌥ P

Pin the answer: scroll, select text, click file:line links, toolbar (copy, read aloud, history)

⌥ O

Open the first file:line in your editor (JetBrains IDEs, Xcode, VS Code, Cursor)

⌥ ↩ / esc

Approve or deny an edit or command (background mode)

esc

Dismiss, or stop the answer

To attach a screenshot to every question instead, turn on VisionCLI menu → Settings… → General → Screenshot every question.

Which project? By default VisionCLI follows the window under your mouse. PhpStorm myshop – payments maps to ~/PhpstormProjects/myshop, and a terminal running claude maps to its folder. With several projects open, pick one in VisionCLI menu → Project. The menu bar shows the current project.

Which session? In Terminal mode (the default), voice questions are typed into the claude (or agy/gemini, see Gemini) already running for that project in iTerm, or a new one is opened. You can type there too; it's one conversation, and answers show in both the terminal and the bubble. Bubble Only mode uses a hidden, leaner session instead. Switch in VisionCLI menu → Settings… → Assistant → Answers appear.

In Claude Code (typing)

After Set Up Claude Code, just talk about what's on screen in any project:

  • "what is this file for?", "why is this view misaligned in the simulator?"

  • "let me select an area", "read the page I have open and summarize it"

  • "how are the orders and payments tables related?"

Slash commands: /mcp__visioncli__see <question>, /mcp__visioncli__point <question>.

Tool

look

Screenshot the window behind the terminal, the one under the mouse, a named app, or a display; reports the pointer position

list_windows, select_region, clipboard_image

Pick a window, drag a box, or use the clipboard image

device_screenshot

Clean iOS Simulator (needs Xcode) or Android (adb) screenshot

db_schema, db_query

Tables, columns, foreign keys; single read-only SELECT/SHOW/DESCRIBE/EXPLAIN/WITH queries in a read-only transaction (MySQL, Postgres, SQLite from .env)

browser_tabs, browser_open, browser_switch_tab, browser_navigate, browser_read, browser_elements, browser_click, browser_type, browser_select, browser_scroll

Fast Chrome/Brave control via AppleScript (dropdowns, toggles with on/off state, custom React/Radix widgets)

browser_fill

Fills a whole form in one call: text fields, dropdowns (native, custom or searchable) and checkboxes

browser_task

Hands a multi-step task on the open page to the Jev agent (about 0.4 s a step)

ui_elements, click, type_text, press_keys, scroll, drag, open_app, open_url

Use any app: Accessibility element list with ids, real mouse/keyboard, by element id or screenshot pixels. Performed by VisionCLI.app (needs Accessibility permission), approved in the bubble

start_voice

Start the voice overlay

Menu

Project (Auto, or pick from running claude/agy/gemini sessions, open editors and recent projects) · Claude Usage · History · New Conversation · Copy Last Answer · Show Terminal · Settings… (⌘,).

Settings… opens one window, laid out like System Settings; changes apply right away:

  • General: start at login, check for updates (now or automatically), Setup Wizard, screenshot every question.

  • Assistant: Claude Code or Gemini (Antigravity), where answers appear (terminal or bubble only), model, effort, fast model for simple questions.

  • Voice: read answers aloud, speed, voice (with a sample), natural voice download, speech recognition (Parakeet, Wispr Flow, Whisper, Apple), vocabulary, microphone and speaker.

  • Computer Control: ask before each action, allow all, or off; Accessibility; AI cursor and instant commands in the browser; watch for app crashes.

  • Meeting Notes: on/off, summaries (colleagues, you), suggested replies, code answers, recording, past meetings, and hiding VisionCLI from screen sharing.

  • Integrations: connect VisionCLI's tools to Claude Code, Antigravity and Gemini CLI; API keys.

  • Advanced: Fix Permissions, log, crash log, home folder.

API keys

VisionCLI menu → Settings… → Integrations → Edit API Keys… opens ~/Library/Application Support/visioncli/integrations.json (readable only by you):

{
  "typesafe": { "apiKey": "..." },
  "gemini": { "apiKey": "..." }
}
  • typesafe: instant browser commands with Jev. Without it, every request goes to the assistant.

  • gemini (optional): makes VisionCLI's Gemini runs use this API key instead of your gemini sign-in.

TYPESAFE_API_KEY in your shell is copied here by visioncli voice if the file has none.

Privacy

  • Screenshots and questions go to the assistant you picked (Claude Code or Antigravity), through your own login.

  • With Jev enabled, quick commands send your spoken words, open tab titles and URLs, and visible link/button text to api.typesafe.ai. Browser tasks also send the visible text, controls and field values of the page in front (never password fields). Turn it off in Settings.

  • Computer control runs locally: the MCP server talks to VisionCLI.app over 127.0.0.1 with a per-launch secret token (in a file only you can read). Nothing is clicked or typed unless an assistant asks and you allow it (or chose Allow All Actions).

  • Whisper and Kokoro run locally. History is stored in ~/Library/Application Support/visioncli/.

Build from source

git clone https://github.com/thethirdsourcers/visioncli.git && cd visioncli
npm install                       # builds the MCP server (dist/)
node dist/index.js install        # register with Claude Code, Antigravity CLI and Gemini CLI (whichever are installed)
npm run build                     # build VisionCLI.app and install it into /Applications
node dist/index.js voice          # start voice mode (under launchd: restarts after a crash)
  • npm run setup-signing (optional, once) creates a local code-signing identity so macOS keeps permissions across rebuilds.

  • npm run release builds the downloadable universal .dmg and .zip into release/. Pushing a v* tag does the same on GitHub Actions and attaches them to a release.

Layout: src/ is the MCP server and CLI (TypeScript). overlay/Sources/ is the menu-bar app (Swift). scripts/ has the build, release and signing scripts.

Troubleshooting

Problem

Fix

"VisionCLI can't be opened"

Right-click → Open, or xattr -dr com.apple.quarantine /Applications/VisionCLI.app

Claude sees only the wallpaper

Voice: enable Screen Recording for VisionCLI. Typing in Claude Code: enable it for your terminal app. Then quit and reopen that app.

Tools missing in Claude Code / Gemini CLI

VisionCLI menu → Settings… → Integrations → Connect / Reconnect (again after moving the app or switching Node versions), then restart claude / gemini

Gemini CLI: "no longer supported for Code Assist for individuals"

Google moved personal accounts to Antigravity: choose VisionCLI menu → Settings… → Assistant → Gemini (Antigravity) (or add a Gemini API key)

Gemini answers "tried a step that needs your approval"

Bubble Only mode can't approve commands/edits; switch VisionCLI menu → Settings… → Assistant → Answers appear to the terminal

Answers only in the terminal, or none at all

VisionCLI menu → Settings… → Advanced → Open Log; check claude is installed and signed in

Microphone errors

Pick a specific mic in VisionCLI menu → Settings… → Voice → Microphone

Sound bars move but no text appears

The pill now says why. Usually: allow Speech Recognition for VisionCLI (Privacy & Security), or turn on Keyboard → Dictation. Installing Whisper also transcribes without Apple's recognizer.

Asked for permissions again after updating, though Settings shows VisionCLI as allowed

The switch belongs to an older signature. VisionCLI 0.5.5+ clears it automatically on first launch; otherwise use VisionCLI menu → Settings… → Advanced → Fix Permissions… (or remove VisionCLI with − and add it again). Happens once when moving to the permanently signed releases; later updates keep permissions.

It crashed

It restarts automatically; details are in VisionCLI menu → Settings… → Advanced → Open Crash Log

Contributing

Contributions are welcome: bug reports, fixes, platform support and docs. See CONTRIBUTING.md for how the code is organised and how to build and test it, and look for issues labelled good first issue to get started. Security problems: SECURITY.md.

License

MIT © Third Sourcers

Available Tools

31 tools
announceTell the user out loudA

Show a short update in VisionCLI's bubble and say it out loud, e.g. when background work you promised to report on has finished ("The v0.7.6 release is live."). One or two short spoken sentences, plain text, no markdown.

Inputs: text (required).

browser_clickClick an elementB

Click an element by id from browser_elements (or by its visible text).

Inputs: target (required).

browser_dragDrag and dropA

Drag an element onto another (ids from browser_elements, which lists draggables, or visible text): sorting lists, boards, upload drop zones, sliders built from divs. Returns the target's text afterwards.

Inputs: to (required), from (required).

browser_elementsList clickable elementsA
Read-only

Visible links, buttons, fields, dropdowns (with options) and toggles (with on/off state) on the active page, with ids (e1, e2, …) for browser_click / browser_type / browser_select.

Inputs: no named parameters.

browser_fillFill in a formA

Fill several fields of a form in one call (ids from browser_elements): text fields get the text, dropdowns (native, custom or searchable) pick the option by its text, checkboxes/radios/switches are set on (true) or off (false). Much faster than one call per field. Each field reports its result; submit with browser_click afterwards.

Inputs: fields (required).

browser_hoverHover over an elementA

Hover over an element (id from browser_elements) to show what appears on hover: menus, captions, tooltips. Then call browser_elements to see and click what appeared.

Inputs: target (required).

browser_navigateBack / forward / reload / close tab / new tabC

Navigate the active tab.

Inputs: action (required).

browser_openOpen a URLA

Open a URL in Chrome/Brave. Always use this (never ⌘L/⌘T + typing) to go to a page: it switches to a tab already showing it, otherwise reuses your own working tab, and only opens one new tab the first time. new_tab: false loads it in the user's current tab.

Inputs: url (required), new_tab.

browser_readRead the current pageA
Read-only

Title, URL, selected text and readable text of the active tab (capped). Faster and more exact than a screenshot.

Inputs: max_chars.

browser_scrollScroll the pageC

Scroll the active page.

Inputs: where (required).

browser_selectChoose a dropdown optionA

Pick an option in a dropdown (native or custom combobox/menu) by its visible text. Use this instead of browser_click for dropdowns. Toggles/switches/checkboxes: use browser_click (browser_elements shows their on/off state).

Inputs: option (required), target (required).

browser_switch_tabSwitch tabB

Bring a tab (id from browser_tabs) to the front.

Inputs: tab_id (required).

browser_tabsList browser tabsA
Read-only

Open tabs in the front-most Brave/Chrome (ids like t1.3 = window 1, tab 3), marking the active one.

Inputs: no named parameters.

browser_taskDo a task on the page (fast)A

Hand a multi-step task on the web page in front of the user's Chrome/Brave to VisionCLI's Jev agent: fill in a form, search this site and open a result, set filters, pick options, add to a cart. It clicks, types and selects in a few seconds (about 0.4 s a step). Put every value to type in the goal: it only types text that's in the goal. Returns each step and where it ended; check the result with browser_read if needed. Open the right page first (browser_open).

Inputs: goal (required).

browser_typeType into a fieldB

Set the value of a form field (id from browser_elements), optionally submitting the form.

Inputs: text (required), target (required), submit.

clickClickA

Click on the screen: by element id from ui_elements (preferred), or at x/y pixels in the last screenshot (from look, or image N of the voice question). Supports right-click and double-click.

Inputs: x, y, count, image, +4 more.

clipboard_imageRead image from clipboardB
Read-only

Return the image currently on the user's clipboard (e.g. a screenshot taken with Cmd+Ctrl+Shift+4).

Inputs: no named parameters.

db_queryRun a read-only SQL queryA
Read-only

Run one SELECT/SHOW/DESCRIBE/EXPLAIN/WITH query against the project's database inside a read-only transaction. Results are capped at 50 rows; use LIMIT.

Inputs: sql (required).

db_schemaDatabase schemaA
Read-only

Tables, columns and foreign keys of the project's database (connection read from the project's .env: Laravel DB_* or DATABASE_URL; MySQL, Postgres, SQLite). Pass a table to narrow it. Read-only.

Inputs: table.

device_screenshotScreenshot a simulator or deviceA
Read-only

Full-resolution screenshot of the booted iOS Simulator (platform ios) or a connected Android device/emulator (platform android), without window chrome. Use it to explain or debug app UI, then find the view in code.

Inputs: platform.

dragDragB

Drag with the left mouse button from (x1,y1) to (x2,y2): move files, select text, resize, reorder.

Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +3 more.

list_windowsList visible windowsA
Read-only

List on-screen app windows (front-most first) with ids, app names and titles. Use before look when unsure which window the user means.

Inputs: no named parameters.

lookLook at the screenA
Read-only

Take a screenshot of what the user is looking at and return it as an image. By default captures the window under the mouse pointer (or, if that is the terminal, the front-most other window). Use target "screen" for the whole display the mouse is on. Optionally pick a window by app name/title or window id, or a display number.

Inputs: app, target, display, window_id, +1 more.

open_appOpen or switch to an appA

Open an app, or bring it to the front if it's running, e.g. "Xcode", "Slack", "Simulator", "Android Studio", "Figma".

Inputs: name (required), look_after.

open_urlOpen a URLB

Open a link or file URL with its default app (browser, Figma, Slack deep links, vscode://, etc.).

Inputs: url (required), look_after.

press_keysPress keysA

Press a key or shortcut (not for browser navigation: use browser_open/browser_navigate), e.g. "enter", "cmd+s", "cmd+shift+p", "tab", "esc", "down". Several separated by spaces run in order: "cmd+a delete".

Inputs: keys (required), app, look_after.

scrollScrollB

Scroll at x/y (default: where the mouse is). dy negative = down, positive = up (lines); dx for sideways.

Inputs: x, y, dx, dy, +3 more.

select_regionLet the user select a regionA
Read-only

Show the native crosshair so the user can drag a rectangle around the exact part of the screen they mean (press Space to pick a window instead, Esc to cancel). Blocks until they finish. Tell the user to make the selection.

Inputs: no named parameters.

start_voiceStart voice modeA

Launch the voice overlay for this project: the user holds Option (Option+Command with a screenshot), asks a question out loud, and the answer appears in a bubble above their mouse pointer.

Inputs: no named parameters.

type_textType textA

Type text with the keyboard into the focused field, or into element (clicked first). replace: true sets the field's whole value instead of typing. Use press_keys for Enter, Tab, shortcuts.

Inputs: text (required), app, element, replace, +1 more.

ui_elementsList UI elementsA
Read-only

List the buttons, fields, menus, links, tabs and text of the front app's focused window (or a named app) via Accessibility, each with an id and its center in screen points. Use the id with click/type_text: it's far more reliable than pixel guessing.

Inputs: app.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.2/5.0

    Scored across 31 tools

    Disambiguation3/5

    Descriptions do distinguish the browser_* family (web pages) from the unprefixed desktop tools, which helps, but there are many near-duplicate pairs: browser_click/click, browser_type/type_text, browser_scroll/scroll, browser_drag/drag, and browser_elements/ui_elements. browser_task also overlaps nearly every other browser tool, so an agent must read carefully to avoid misselection.

    Naming Consistency3/5

    All names use snake_case, but conventions are mixed: browser_* tools are consistent, while desktop tools swap between noun-only (look), verb (click, scroll, drag), and verb_noun (type_text, press_keys, open_app, open_url). The browser_ prefix scoping is a plus, yet the unprefixed half isn't predictable.

    Tool Count3/5

    31 tools exceeds the comfortable 15-tool band and sits in the 'heavy' range, but the server genuinely spans browser automation, native app control, database inspection, and device screenshots, so most tools earn their place. A few (e.g. the browser_task meta-tool plus individual browser actions) feel redundant rather than strictly necessary.

    Completeness4/5

    The surface covers a broad lifecycle: page discovery, click/type/select/drag/scroll, tab and navigation control, plus desktop input and app launching, with db read and device screenshots. Gaps are minor (db is read-only, no explicit browser_close/tab management beyond switch), and agents can work around them.

    Maintenance

    ActivityMaintained
    ResponsivenessMostly unanswered

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Enables AI agents to capture and analyze screenshots of macOS applications, windows, or the entire screen using local (Ollama) or cloud-based AI vision models, with non-intrusive, fast screen capture via Apple's ScreenCaptureKit.
      3
      23 npm
      2
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Screenshot and diagram tool for AI agents. Capture and annotate screenshots to show Claude what you mean — or let the agent render Mermaid diagrams and open them for visual review. Approve, annotate, or request changes with text feedback. Built-in review mode with structured responses. CLI and MCP server for Claude Code, Cursor, Windsurf, Cline. macOS, open source, free.
      5 npm
      352
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Provides AI coding assistants with privacy-first, synchronized screen context including cursor position, active window metadata, and visual verification to aid in building and debugging graphical interfaces.
      MIT