Skip to main content
Glama

Touchpoint is a cross-platform Python library for reading and interacting with desktop UI through native accessibility APIs. One import, one API — works on Linux, macOS, and Windows, with built-in support for Chromium and Electron apps via CDP (Chrome DevTools Protocol).

Instead of scraping pixels or running vision models, Touchpoint reads the real accessibility tree — structured names, roles, states, and positions for every element on screen. Fast and reliable, with no vision model required. Ships with an MCP server so LLM agents like Claude, Cursor, or any local model can control any desktop app out of the box.

import touchpoint as tp

elements = tp.find("Send", role=tp.Role.BUTTON, app="Slack")
tp.click(elements[0])

Why Touchpoint?

Screenshot / vision

Browser automation

Touchpoint

Native desktop apps

⚠️ inaccurate or slow

❌

✅

Browsers

⚠️ inaccurate or slow

✅

✅ via CDP

Electron apps (Slack, VS Code, ...)

⚠️ inaccurate or slow

⚠️ web content only

✅ native + web

Structured element data

❌ needs OCR/vision model

✅ web only

✅ names, roles, states, positions

Works with local / non-vision models

❌

✅ web only

✅ all apps

Works across Linux, macOS, Windows

✅

✅

✅


Table of Contents


Related MCP server: servo-mcp

Install

Requires Python 3.10+.

pip install touchpoint-py

Everything is included: your platform's native backend, CDP support for browsers and Electron apps, the MCP server, and screenshot capabilities. Platform-specific dependencies are installed automatically via pip environment markers.

Platform requirements

Platform

Backend

Requirement

Linux

AT-SPI2

Install xdotool (required for input + minimize_window) and wmctrl (required for all window management — used for AT-SPI → X11 id mapping). Most desktops include python3-gi and gir1.2-atspi-2.0 — install them if missing.

Windows

UI Automation

None — uses built-in COM APIs

macOS

Accessibility (AX)

Grant permission: System Settings → Privacy & Security → Accessibility


Quick Start

import touchpoint as tp

# Discover
apps = tp.apps()                            # ["Firefox", "Slack", "Terminal", ...]
windows = tp.windows()                      # Window objects with title, position, size
all_els = tp.elements(app="Firefox", named_only=True)  # only elements with text labels

# Find
results = tp.find("Search", role=tp.Role.TEXT_FIELD, app="Firefox")

# Act
tp.set_value(results[0], "touchpoint python", replace=True)
tp.press_key("enter")
tp.hotkey("ctrl", "s")                      # keyboard shortcuts

# Wait for UI changes
tp.wait_for("results", app="Firefox", timeout=10)

# Screenshot
img = tp.screenshot()                       # full desktop → PIL.Image
img = tp.screenshot(app="Firefox")           # cropped to app window

Element IDs

Every element has a unique ID like atspi:1234:1:2.0 or cdp:9222:TID:4. Action functions accept either an Element object or a bare ID string — useful for storing references across steps:

results = tp.find("Send", max_results=1)
element_id = results[0].id                  # "atspi:1234:1:5.2"

# later...
tp.click(element_id)                        # works with just the string

Output formats

Control how results are returned:

tp.elements(app="Slack", format="flat")     # one compact line per element (best for LLMs)
tp.elements(app="Slack", format="tree")     # indented parent/child hierarchy
tp.elements(app="Slack", format="json")     # full JSON with all fields

MCP Server

Touchpoint ships an MCP (Model Context Protocol) server ready for any MCP-compatible client. Use it to let LLM agents like Claude, Cursor, local models, or any tool that supports MCP control your desktop.

Two modes — vision and no-vision

Set TOUCHPOINT_MODE=no-vision (default: vision) to switch modes:

  • Vision mode — agents use screenshot() to see the screen and interact by element ID or coordinates. Best for frontier models with strong vision capabilities.

  • No-vision mode — agents use snapshot() to get a compact structured text tree of the active window, then act on element IDs directly. Works with any model including local ones that have no vision capability. Most action tools append auto-verify flags ((new window: ...), (focus moved), (no change detected)) so the agent can detect state changes without taking a screenshot.

Tools

Category

Vision mode

No-vision mode

Orient

screenshot, snapshot, apps, windows

snapshot, diff_snapshot, apps, windows

Find

find, get_element

find

Read

read_text

read_text

Actions

click (element or coordinates), set_value, set_numeric_value, select_text, focus, action

click (element only), set_value, set_numeric_value, select_text, focus, action

Keyboard

type_text, press_key

type_text, press_key

Mouse

mouse_move, scroll

scroll

Window

activate_window, minimize_window, fullscreen_window, close_window, move_window, resize_window

activate_window, minimize_window, fullscreen_window, close_window

Waiting

wait_for, wait_for_app, wait_for_window

wait_for, wait_for_app, wait_for_window

Health

diagnostics

diagnostics

The MCP server includes built-in instructions that teach agents the correct workflow for each mode — including the orient → act → verify loop, when to use read_text vs find, and how to recover from errors.

         ┌──────────┐
    ┌───▶│  ORIENT  │  screenshot · apps · windows
    │    └────┬─────┘
    │         ▼
    │    ┌──────────┐
    │    │  LOCATE  │  find · snapshot · get_element
    │    └────┬─────┘
    │         ▼
    │    ┌──────────┐
    │    │   ACT    │  click · set_value · type_text · press_key
    │    └────┬─────┘
    │         ▼
    │    ┌──────────┐
    │    │  VERIFY  │───▶ Done ✅
    │    └────┬─────┘
    │         │ not yet
    └─────────┘

Client setup

Config file location:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

  • Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "touchpoint": {
      "command": "touchpoint-mcp"
    }
  }
}

If using a virtualenv, use the full path: "/path/to/venv/bin/touchpoint-mcp"

Add to .vscode/mcp.json in your workspace:

{
  "servers": {
    "touchpoint": {
      "command": "touchpoint-mcp"
    }
  }
}

Create or edit ~/.cursor/mcp.json:

{
  "mcpServers": {
    "touchpoint": {
      "command": "touchpoint-mcp"
    }
  }
}

Edit ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "touchpoint": {
      "command": "touchpoint-mcp"
    }
  }
}
claude mcp add touchpoint -- touchpoint-mcp

Add to mcpServers in ~/.openclaw/openclaw.json:

{
  "mcpServers": {
    "touchpoint": {
      "command": "touchpoint-mcp"
    }
  }
}

Environment variables

Variable

Example

Description

TOUCHPOINT_CDP_DISCOVER

true

Auto-discover CDP ports from running processes

TOUCHPOINT_CDP_PORTS

{"Chrome": 9222}

Explicit app-to-port mapping (JSON)

TOUCHPOINT_CDP_APP

Google Chrome

Single app name (pair with _PORT)

TOUCHPOINT_CDP_PORT

9222

Single port (pair with _APP)

TOUCHPOINT_CDP_REFRESH_INTERVAL

5.0

Seconds between CDP port scans

TOUCHPOINT_SCALE_FACTOR

1.25

Display scale override

TOUCHPOINT_FUZZY_THRESHOLD

0.6

Minimum match score for find() (0.0–1.0)

TOUCHPOINT_FALLBACK_INPUT

true

Use coordinate fallback when native actions fail

TOUCHPOINT_MAX_ELEMENTS

5000

Maximum elements per query

TOUCHPOINT_MAX_DEPTH

20

Default tree depth limit

TOUCHPOINT_AX_MESSAGING_TIMEOUT

1.0

Max seconds to wait for a macOS AX app reply


Browser & Electron Apps (CDP)

Native accessibility APIs return limited data for Electron and Chromium apps (Slack, Discord, VS Code, etc.). Touchpoint's CDP backend connects via Chrome DevTools Protocol to get the full web content.

Auto-discovery is enabled by default — Touchpoint automatically finds running browsers and Electron apps that were launched with a debug port. No manual configuration needed beyond launching the app with the flag.

Setup

  1. Launch the app with a debug port:

# Linux
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/tp-chrome

# macOS
open -na "Google Chrome" --args --remote-debugging-port=9222 --user-data-dir=/tmp/tp-chrome

# Windows
start chrome --remote-debugging-port=9222 --user-data-dir=%TEMP%\tp-chrome
  1. Configure Touchpoint:

import touchpoint as tp

tp.configure(cdp_discover=True)             # auto-discover from running processes
# or
tp.configure(cdp_ports={"Google Chrome": 9222})  # explicit mapping
  1. Control what you get with the source parameter:

tp.elements(app="Google Chrome", source="full")     # native chrome + web content (default)
tp.elements(app="Google Chrome", source="cdp_ax")   # web content only (CDP accessibility tree)
tp.elements(app="Google Chrome", source="native")   # native UI only (toolbar, tabs, menus)
tp.elements(app="Google Chrome", source="dom")      # DOM walker (catches what AX misses)

CDP results are merged with native backend results — you get the toolbar and window controls from AT-SPI2/UIA/AX, combined with the full web page content from CDP, in a single elements() call.

source="ax" remains accepted as a compatibility alias for source="cdp_ax". Prefer cdp_ax in new code so it is not confused with the native macOS AX backend.


API Reference

Discovery

Function

Description

tp.apps()

List application names in the accessibility tree

tp.windows()

All windows with id, title, app, position, size, active state

tp.elements(app, role, states, ...)

UI elements, with filtering, tree mode, and formatting

tp.element_at(x, y)

Deepest element at screen coordinates

tp.get_element(id)

Fresh snapshot of a single element by ID

Search & Wait

Function

Description

tp.find(query, app, role, ...)

Search by name — 4-stage matching: exact → contains → word → fuzzy

tp.wait_for(query, ...)

Poll until elements appear (or disappear with gone=True)

tp.wait_for_app(app, ...)

Poll until an app appears or disappears

tp.wait_for_window(title, ...)

Poll until a window appears or disappears

Actions

Function

Description

tp.click(element)

Click via accessibility action, with coordinate fallback

tp.double_click(element)

Double-click

tp.right_click(element)

Right-click / context menu

tp.set_value(element, text)

Set text content (replace=True to clear first)

tp.set_numeric_value(element, n)

Set slider or spinbox value

tp.select_text(element, text)

Select a substring within text content across Linux, Windows, macOS, and web/CDP

tp.select_text_range(element, start, end)

Select a character range when you already know the offsets

tp.focus(element)

Move keyboard focus

tp.action(element, name)

Execute a raw accessibility action by name

tp.activate_window(window)

Bring a window to the foreground (restores from minimized)

tp.minimize_window(window)

Minimize a window. Use activate_window to restore.

tp.fullscreen_window(window, fullscreen=True)

Enter or exit fullscreen for a window

tp.close_window(window)

Politely close a window

tp.move_window(window, x, y)

Move a window to a new screen position

tp.resize_window(window, width, height)

Resize a window to width × height pixels

Input

Function

Description

tp.type_text(text)

Type into the currently focused element

tp.press_key(key)

Press and release a key ("enter", "tab", "escape")

tp.hotkey(*keys)

Key combination (tp.hotkey("ctrl", "s"))

tp.click_at(x, y)

Click at screen coordinates

tp.double_click_at(x, y)

Double-click at coordinates

tp.right_click_at(x, y)

Right-click at coordinates

tp.mouse_move(x, y)

Move the cursor

tp.scroll(direction, amount)

Scroll at current cursor position

Screenshot & Config

Function

Description

tp.screenshot(app, element, ...)

Full desktop or cropped to app/window/element/monitor

tp.monitor_count()

Number of connected monitors

tp.configure(...)

Set runtime options (see Configuration)

tp.diagnostics()

Report backend, input, CDP, timeout, and dependency health

All action functions accept an Element object or a string ID. elements(), find(), and get_element() support format="flat", format="json", or format="tree" (elements only) to return pre-formatted strings instead of objects. Window management is implemented across Linux AT-SPI2, Windows UIA, and macOS AX backends.


Architecture

┌───────────────────────────────────────────────────────┐
│               import touchpoint as tp                 │
│  tp.find() · tp.click() · tp.screenshot() · ...       │
│                    (Public API)                       │
├─────────────────────────┬─────────────────────────────┤
│     Backend (ABC)       │    InputProvider (ABC)      │
├─────────────────────────┼─────────────────────────────┤
│  AT-SPI2     (Linux)    │  Xdotool       (X11)        │
│  UIA         (Windows)  │  SendInput     (Win32)      │
│  AX          (macOS)    │  CGEvent       (macOS)      │
│  CDP         (browsers) │                             │
├─────────────────────────┴─────────────────────────────┤
│  Utilities: formatter · matcher · screenshot · scale  │
└───────────────────────────────────────────────────────┘

Two-layer design:

  • Backend reads the accessibility tree and runs structured actions (click, set_value, focus). Element-aware and reliable.

  • InputProvider simulates raw keyboard and mouse input. Coordinate-based and element-blind. Used as an automatic fallback when a native accessibility action isn't available.

CDP runs alongside the platform backend. Their results are merged: native window chrome (toolbar, tabs, menus) from AT-SPI2/UIA/AX, plus full web content from CDP, unified under one API.

For detailed internals, see ARCHITECTURE.md.


Configuration

tp.configure(
    fuzzy_threshold=0.6,          # minimum match score for find() (0.0–1.0)
    fallback_input=True,          # use InputProvider when native actions fail
    type_chunk_size=40,           # split long text into chunks for typing (0 = disable)
    max_elements=5000,            # max elements per query
    max_depth=20,                 # default tree depth limit
    scale_factor=None,            # display scale override (None = auto-detect)
    cdp_ports={"Chrome": 9222},   # explicit CDP port mapping
    cdp_discover=True,            # auto-discover CDP ports from running processes
    cdp_refresh_interval=5.0,     # seconds between CDP target scans
    ax_messaging_timeout=1.0,     # max seconds to wait for a macOS AX app reply
)

tp.diagnostics() returns a JSON-friendly health report. It includes the active backend, input provider, CDP targets, optional platform tools, configured timeouts, and macOS apps recently skipped after an AX messaging timeout.


Development

git clone https://github.com/Touchpoint-Labs/touchpoint.git
cd touchpoint
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest

Status

Alpha — fully functional and tested on all three platforms. The API may change before 1.0 based on user feedback.

Platform

Backend

Input

CDP

Tests

Linux (X11)

✅ AT-SPI2

✅ xdotool

✅

✅

Windows

✅ UIA

✅ SendInput

✅

✅

macOS

✅ AX

✅ CGEvent

✅

✅

Known limitations

  • Wayland input — The Linux InputProvider uses xdotool, which requires X11. On pure Wayland (no XWayland), keyboard/mouse simulation is unavailable. The accessibility tree and native actions still work.

  • Synchronous CDP — CDP calls block on WebSocket responses. JavaScript dialogs (alert, confirm, prompt) are auto-dismissed to prevent deadlocks. An async rewrite is planned.

  • No browser navigation API — Touchpoint doesn't have built-in URL navigation. Agents can navigate by interacting with UI elements directly: find the address bar, type a URL, press Enter.

  • CDP windows are page targets, not OS windows — but window management still works: tp.activate_window() brings the target forward via CDP, and minimize/fullscreen/close/move/resize on a surfaced cdp: window are routed to the underlying native OS window (resolved by owning PID) and handled by the platform backend. They raise ActionFailedError only if no native OS window for that target can be found (e.g. it has been closed).

  • Backend role/state parity is still uneven — macOS AX and Windows UIA both improved significantly in 0.3.0, but Windows still relies on more heuristics and has more unmapped long-tail roles than the other backends.


Roadmap

High Priority

  • Async CDP architecture — non-blocking WebSocket, proper dialog queuing, concurrent multi-tab queries

Medium Priority

  • Backend role/state parity — close remaining role mapping gaps, especially UIA long-tail roles on Windows

  • Wayland input backend — libei / xdg-desktop-portal RemoteDesktop when X11 isn't available

Lower Priority

  • Tooltip and notification visibility

  • Element caching


License

MIT

Available Tools

27 tools
actionA

Perform a raw accessibility action by exact name.

Only call this when the element shows actions=[...] in snapshot()
output AND click() either failed silently or is not the right verb.
Use the exact action name shown in the snapshot — do not guess.
Example: an element showing actions=[expand, collapse] can be acted
on with action(element_id, "expand").

Args:
    element_id: The element ID.
    action_name: Exact action name from the snapshot actions=[...] list.

Inputs: element_id (required), action_name (required). Returns: result

activate_windowA

Bring a window to the foreground.

Element-scoped actions (click, set_value, select_text, etc.) bring
their window to focus automatically — you rarely need this.  Use it
when you need to type_text or press_key in a window without first
clicking an element in it (e.g. sending a keyboard shortcut to a
window you haven't interacted with yet in this task).

Args:
    window_id: The window ID to activate (from windows()).

Inputs: window_id (required). Returns: result

appsA

List applications with accessible UI elements.

Returns application names visible in the accessibility tree. Use these names to scope other tools (find, elements, screenshot).

Inputs: no named parameters. Returns: result

clickA

Click an element by ID, or at screen coordinates.

Pass ``element_id`` to click via the element's native
accessibility action (most reliable).  Pass ``x`` and ``y``
to click directly at screen coordinates instead — useful
when clicking by ID triggers an unintended action (e.g.
opens a dropdown instead of focusing a text entry).
Every element shows its position as @(x,y) in listings.
Coordinate clicks always report OK even if nothing was hit —
verify the result with a screenshot or find().

Args:
    element_id: The element ID to click.
    x: Screen X coordinate (use with y instead of element_id).
    y: Screen Y coordinate (use with x instead of element_id).
    button: "left" (default) or "right".
    double_click: If true, perform a double-click instead.
        Cannot be combined with button="right".

Inputs: element_id, x, y, button, +1 more. Returns: result

close_windowB

Close a window.

Use windows() to find the window ID first.

Args:
    window_id: The window ID to close.

Inputs: window_id (required). Returns: result

diagnosticsA

Report backend, input, CDP, and optional dependency health.

Args:
    probe: If true (default), initialise optional providers and perform
        discovery. Pass false for a side-effect-free state snapshot.

Inputs: probe. Returns: result

findA

Search for UI elements by name — buttons, links, labels, fields.

Matches against element names (button labels, link text, field
labels), NOT body text or prose content inside articles or
documents.

Returns element IDs that you can use with click, set_value, etc.
Use the FULL visible text for best results (e.g. "Send Message"
not just "Send").

Args:
    query: Text to search for (e.g. "Send Message", "Submit", "Search").
    app: Scope to this application (e.g. "Firefox", "Slack").
    window_id: Scope to this window.
    role: Only match this role (e.g. "button", "text_field", "link").
    states: Only match elements with ALL these states (e.g. ["enabled", "visible"]).
    max_results: Maximum matches to return.
    fields: Which fields to search -- ["name"], ["name", "value"], or ["name", "value", "description"].
    source: "full" (default, merged native+web), "cdp_ax" (CDP accessibility tree only), "native" (platform only), or "dom" (live DOM). "ax" remains as a compatibility alias for "cdp_ax".

Inputs: query (required), app, window_id, role, +4 more. Returns: result

focusB

Move keyboard focus to an element.

Args:
    element_id: The element ID to focus.

Inputs: element_id (required). Returns: result

fullscreen_windowA

Enter or exit fullscreen for a window.

Use windows() to find the window ID first.

Args:
    window_id: The window ID.
    fullscreen: True to enter fullscreen (default), False to exit.

Inputs: window_id (required), fullscreen. Returns: result

get_elementA

Get a single element by its ID with full detail.

Returns a fresh snapshot with current states, value, supported
actions, and description.  Use this to inspect an element
before calling the ``action()`` tool — the actions list shows
exactly which raw action names are available.

Args:
    element_id: The element ID (from find/elements results).

Inputs: element_id (required). Returns: result

minimize_windowA

Minimize a window to the dock/taskbar.

Use windows() to find the window ID first.
To restore a minimized window, use activate_window().

Args:
    window_id: The window ID to minimize.

Inputs: window_id (required). Returns: result

mouse_moveA

Move the mouse cursor to an element or to screen coordinates.

Use this before scroll() to scroll within a specific area.

Args:
    element_id: The element ID to move the cursor to.
    x: Screen X coordinate (use with y instead of element_id).
    y: Screen Y coordinate (use with x instead of element_id).

Inputs: element_id, x, y. Returns: result

move_windowB

Move a window to a new screen position.

Use windows() to find the window ID first.

Args:
    window_id: The window ID to move.
    x: New horizontal position of the top-left corner.
    y: New vertical position of the top-left corner.

Inputs: window_id (required), x (required), y (required). Returns: result

press_keyA

Press a key or key combination.

Single key: "enter", "tab", "escape", "f5", "backspace".
Combination: ["ctrl", "s"], ["ctrl", "shift", "p"], ["alt", "f4"].

Args:
    keys: A single key name, or a list of keys for a combination
          (all held together, then released in reverse order).
    repeat: Number of times to press (default 1).

Inputs: keys (required), repeat. Returns: result

read_textA

Return text content exposed by an element or container.

Some backends expose aggregate descendant text for containers.
Pass a container ID from snapshot() to read an entire section,
article, dialog, or document body when the backend supports it.

Workflow: snapshot() to find the right container, read_text(id)
to read its full contents.

Args:
    element_id: The element or container ID to read text from.

Inputs: element_id (required). Returns: result

resize_windowA

Resize a window.

Use windows() to find the window ID first.

Args:
    window_id: The window ID to resize.
    width: New width in pixels.
    height: New height in pixels.

Inputs: window_id (required), width (required), height (required). Returns: result

screenshotA

Capture the screen and return an image.

With no arguments, captures the full desktop.  Specify one
parameter to crop to a specific target.

Args:
    app: Crop to this application's window.
    window_id: Crop to this specific window.
    element_id: Crop to this element's bounding box.
    padding: Extra pixels around the crop region.
    monitor: Capture only this monitor (0-indexed).

Inputs: app, window_id, element_id, padding, +1 more.

scrollA

Scroll by mouse wheel ticks.

With ``element_id``, moves the cursor to that element's position
first, then scrolls — useful for scrolling inside a specific list,
panel, or container.  Without it, scrolls at the current cursor
position.

Args:
    direction: One of "up", "down", "left", "right".
    amount: Number of scroll ticks (default 3).
    element_id: Optional element to scroll at.  When given, cursor
        is moved to the element's center first.

Inputs: direction (required), amount, element_id. Returns: result

select_textA

Select a substring within an element's text content.

Finds the text within the element and applies a native text
selection over that range.  Works on editable fields and document
bodies on all backends.  On web content (CDP) and Windows (UIA)
also works on read-only containers such as articles and sections.
Useful for formatting, copying, or replacing specific text.

Args:
    element_id: The element ID containing the text.
    text: The exact substring to select.
    occurrence: Which occurrence to select (1 = first, 2 = second, etc.).

Inputs: element_id (required), text (required), occurrence. Returns: result

set_numeric_valueA

Set the numeric value of a range element (slider, spinbox).

Args:
    element_id: The element ID (a slider, spin button, etc.).
    value: The numeric value to set.

Inputs: element_id (required), value (required). Returns: result

set_valueA

Set the value of an editable element.

Calls the platform's native value-setting API.  Behaviour depends
on the target widget:
  * Text fields / text areas: writes the text.
  * Combo boxes / dropdowns / <select>: picks the option whose
    label or value matches *value*.  Prefer this over clicking
    the dropdown and then clicking a popup item — picking by name
    is one atomic call and avoids popup-click failures on some
    toolkits.
  * Other editable widgets: whatever their value interface accepts.

Args:
    element_id: The element ID (a text field, combo box, etc.).
    value: The text or option label to set.
    replace: If true, clear the field first and replace all content.
             If false (default), insert at the current cursor position.

Inputs: element_id (required), value (required), replace. Returns: result

snapshotA

Return a tree-structured view of a window's UI.

The primary orient tool in no-vision mode.  In vision mode, use it
as a structural complement to screenshot() — cheaper than a
screenshot and gives element IDs directly.  Default behaviour: pick
the currently active window, walk its accessibility tree, prune
anonymous structural wrappers, preserve semantic containers (dialogs,
menus, lists, etc.), and emit an indented text view with one line per
interactive element.

Args:
    app: Snapshot the given app's active window (or first window).
        Case-insensitive.
    window_id: Snapshot a specific window by ID.
    element_id: Start the tree walk from a specific element instead
        of the window root.  Use to dig into a container whose
        children were not visible in a previous (truncated) snapshot.
        To read the text content of a container, use read_text()
        rather than snapshot().
    all_elements: If true, include every named element — not just
        interactive + container roles.  Use when the default
        filter is hiding something.
    max_depth: Maximum tree depth to walk.  Defaults to the
        configured value (typically 20).  Decrease for a faster
        overview of a large window.

Inputs: app, window_id, element_id, all_elements, +1 more. Returns: result

type_textA

Type text into the currently focused element.

Simulates keyboard input.  Focus a text field first with
click() or focus(), then type into it.

Special characters:
  \n = Enter (line break),  \t = Tab (next field),
  \b = Backspace (delete previous character).

Args:
    text: The text to type.
    raw: If true, type literal backslashes without converting
        ``\n``, ``\t``, or ``\b`` escape sequences.

Inputs: text (required), raw. Returns: result

wait_forA

Wait for elements to appear or disappear.

Polls until matching elements are found (or gone) or timeout.
Use after actions that trigger UI changes.

Args:
    element: Text to search for.  Pass a single string (e.g.
        "Submit") or a list of strings (e.g. ["Success", "Error"])
        for multi-query mode.  With mode="any", returns as soon
        as any query matches.  With mode="all", waits until every
        query has matched.
    app: Scope to this application.
    window_id: Scope to this window.
    role: Only match this role.
    states: Only match elements with ALL these states.
    fields: Which fields to search (default: ["name"]).
    mode: "any" (return when any query matches) or "all"
        (wait for all queries to match).  Only meaningful when
        element is a list.
    timeout: Maximum seconds to wait (default 10).
    source: "full" (default), "cdp_ax", "native", or "dom".
        "ax" remains as a compatibility alias for "cdp_ax".
    max_results: Maximum elements to return (default 5).
    wait_for_new: If true, ignore elements already present -- wait for NEW ones.
    gone: If true, wait for matching elements to DISAPPEAR instead.

Inputs: element (required), app, window_id, role, +8 more. Returns: result

wait_for_appA

Wait for an application to appear or disappear.

Polls the application list until the app is found (or gone).
Use after launching or closing an application.

Args:
    app: Application name to wait for (e.g. "Firefox", "Slack").
    timeout: Maximum seconds to wait (default 10).
    gone: If true, wait for the app to DISAPPEAR instead.

Inputs: app (required), timeout, gone. Returns: result

wait_for_windowA

Wait for a window to appear or disappear.

Polls the window list until a window with a matching title is
found (or gone).  Use after actions that open or close windows.
When found, auto-activates the window so subsequent type_text
and press_key target it.

Args:
    title: Window title to search for (substring match).
    app: Only look for windows in this application.
    timeout: Maximum seconds to wait (default 10).
    gone: If true, wait for the window to DISAPPEAR instead.

Inputs: title (required), app, timeout, gone. Returns: result

windowsA

List all open windows.

Returns window IDs, titles, sizes, and app names.
Use window IDs to scope find/elements queries or activate_window.

Inputs: no named parameters. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 27 tools

    Disambiguation5/5

    Each tool targets a distinct resource and action: discovery (apps, windows, find, snapshot, screenshot), element inspection (get_element), input (click, set_value, set_numeric_value, focus, select_text, type_text, press_key), and window management (activate, minimize, close, move, resize, fullscreen). Overlaps like find vs wait_for are differentiated by polling behavior, and mouse_move/scroll have distinct roles.

    Naming Consistency5/5

    All tools follow a consistent lower_snake_case verb_noun pattern (mouse_move, minimize_window, activate_window, set_value, wait_for_window, etc.). Exceptionally uniform, with no mixed conventions or vague verbs. Even atypical names like 'find', 'snapshot', and 'action' function as clear verbs.

    Tool Count4/5

    At 27 tools, the count exceeds the typical 3-15 range, but the domain (cross-platform UI automation) justifies the breadth. Each tool covers a distinct operation—discovery, inspection, input, window manipulation, and waiting—with no redundancy. The count feels justified by the complexity required to handle various platforms and web content.

    Completeness5/5

    The surface is remarkably complete for UI automation: it covers discovery (apps/windows/elements), inspection (snapshot, get_element, read_text), interaction (click, type, set_value, scroll, drag via coordinate/action), and synchronization (wait_for, wait_for_app, wait_for_window). The generic `action` tool covers any raw accessibility action, providing a fallback for unforeseen operations. No critical dead-ends.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to see and control your desktop with tools for screenshots, clicks, typing, and more, all locally on macOS and Windows.
      107 npm
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Lets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.
      25
      MIT