Press a UI control directly through the OS accessibility tree (macOS Accessibility; Windows UI Automation is not yet implemented and returns a clear error) — no coordinates, no cursor movement, works in the background (the app need not be frontmost). query is a case-insensitive substring of the element's accessibility role or label/title (e.g. "Send", "Search"). Targets the last window-captured app (or the frontmost app). Only works for apps that expose an accessibility tree; if it returns "no element matching" (or the not-implemented error), fall back to screenshot + left_click.

Move keyboard focus to a control through the OS accessibility tree (macOS Accessibility; not yet implemented on Windows, where it returns a clear error). Background, no cursor. query matches the element's role/label. Targets the last window-captured app (or frontmost). Native-app accessibility only.

Set a control's value directly through the OS accessibility tree (macOS Accessibility; not yet implemented on Windows, where it returns a clear error) — e.g. fill a text field without focusing or typing. Background, no cursor. query matches the element's role/label; value is the text to set. Targets the last window-captured app (or frontmost). Native-app accessibility only.

Execute a sequence of input actions (mouse_move, left_click, right_click, double_click, scroll, key_combo, type_text, wait) in one call to reduce round-trips. Coordinates are in screenshot space. Take a screenshot separately afterwards to observe the result.

Activate an actionable element by the mark NUMBER shown in the most recent screenshot(marks=true) — the reliable way to click without guessing coordinates. Always background, no cursor movement: web-page content in a scriptable browser (Safari, Chrome, Arc, Edge, Brave, …) is clicked through the page's OWN JavaScript engine (an Accessibility press is a silent no-op on web content), native controls through the Accessibility tree; if neither applies it falls back to a click at the element's center. Numbers go stale when the UI changes, so take a fresh screenshot(marks=true) right before calling this. If the number is unknown, re-shoot with marks=true.

Get the current mouse cursor position. Returns (x, y) in logical coordinates.

Double-click at the given (x, y) coordinates (in screenshot space).

DEBUG: dump the target app's Accessibility tree (roles, subroles, labels, actions, frames) as indented text — to diagnose why some elements are not marked. Targets the last window-captured app (or frontmost).

Simulate a key combination (e.g., "cmd+c", "shift+tab").

Left-click at the given (x, y) coordinates (in screenshot space).

List all installed applications on the system.

List all visible windows across all applications.

Move the mouse cursor to the given (x, y) coordinates (in screenshot space).

Read on-screen TEXT via the platform OCR engine (Apple Vision on macOS, Windows.Media.Ocr on Windows). Captures the display (or window="") and returns the recognized text lines, each with a clickable center in the same pixel space as a screenshot — so you can both READ the text and click a line with left_click(x, y). Returns text only (no image), so it is a cheap, fast way to pull text off the screen. Best when you need to read or click TEXT on a surface where marks come back empty — canvas, games, image-rendered or custom-drawn views — or to grab a lot of text at once without parsing a screenshot. For native/web UI with an Accessibility tree, screenshot(marks=true) + click_mark is still more precise. Languages default to Simplified Chinese + English; pass languages=[...] (BCP-47) to override.

Launch or focus an application by name (e.g., "Safari", "Slack").

Read the current system clipboard contents as text.

Right-click at the given (x, y) coordinates (in screenshot space).

Capture the screen — the whole main display, or a single window with window="" — and return a base64 JPEG plus a text note with its pixel dimensions. ALL coordinate-taking tools (mouse_move, *_click, scroll) expect coordinates in THIS image's pixel space — origin (0,0) top-left, x right, y down — so read target positions directly off the returned image; subsequent clicks are mapped through it automatically. PREFER window="" (substring of its title or app name) over the whole display whenever you are working inside one app: a full-display shot is downscaled to ~1280px wide, so small UI (list rows, sidebar items, buttons) becomes only a few pixels — too small to read or click accurately. A window capture is larger and sharper, returns a smaller image (fewer pixels → less context, faster turn), and clicks map into the window automatically. marks is ON by default: it boxes+numbers actionable elements (needs Accessibility) and lists each as [N] — activate one with click_mark(number=N), the most reliable way to click with no coordinate guessing. A magenta coordinate grid (for reading x/y) is shown automatically when marks is off and hidden when it is on; pass grid=true to force both, or marks=false for pure coordinate mode. If a target is still too small to click, use zoom_region to magnify part of this image.

Scroll at the given (x, y) position. Positive lines = up, negative = down.

Type a string of text into the currently focused element.

Wait for a specified number of seconds before returning.

Write text to the system clipboard.

Zoom into a rectangle of the CURRENT image (the last screenshot's pixel space) and re-capture it at native resolution — a sharp, legible magnified view. Only that rectangle is captured (not the whole display), so it is also smaller and quicker to take than a full-display shot. Use it to read exact positions on surfaces that expose no marks (canvas/games, custom-rendered views) before clicking, or to stay scoped while working inside one area. Pass x, y, width, height in the current image's pixels (width,height > 0). The returned image becomes the new coordinate space, and clicks afterward map into the zoomed region automatically. marks defaults OFF (read coordinates off the overlaid grid); grid defaults ON. Take a screenshot first so there is an image to zoom into.