iphone-use
This MCP server lets AI agents and scripts inspect and control a real iPhone through a macOS daemon and on-device XCTest runner, plus manage deterministic flows and app/device debugging.
Read the current iPhone screen as an accessibility text tree, screenshots, or a live MCP Apps panel; query elements, find labels, and read field values.
Perform UI actions: tap by coordinates/label/element, type Unicode text, press keys, swipe/scroll, drag, long-press, multi-touch, pickers, and system alerts.
Batch multi-step actions in one call with validation and
wait_forconditions, returning the settled screen; collect scrolling lists across pages.Discover apps by name and bundle ID, launch apps/deep links, install
.ipa/.appbuilds, set simulated GPS location, and read/write per-app notes.List, inspect, run, draft, publish, update, and report issues for deterministic per-app flows that replay without a model.
Inspect phone status/capabilities/instances, reconnect when needed, hold/release the phone lease, and respect privacy/setup blockers.
Debug apps with live system logs, crash reports, and screen recording to MP4.
Support human remote control via browser/iOS app and MCP Apps live panel, with one owner at a time and safety controls.
Enables AI agents to control a real iPhone running iOS 15 or later, reading the screen as an accessibility tree and performing taps, typing, and swipes across any app, including those without an API.
Allows agents to interact with WeChat on a real iPhone, such as finding and reading messages, by driving the WeChat UI through screen reading and touch automation.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@iphone-useOpen Health and tell me how many steps I walked today."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
curl -fsSL https://raw.githubusercontent.com/leeguooooo/iphone-use/main/install.sh | shhttps://github.com/user-attachments/assets/ee298774-bf3e-4fd0-9a08-481b20904f61
1-minute demo, real iPhone recordings · 中文版
Advertisement · Available
Advertise here
Showcase your product here and support ongoing maintenance.
Ask about advertising · leeguoooo@gmail.com
Paid setup and project sponsorship
Need help setting up a real iPhone, connecting your agent or creating an iOS app test flow? Email us with your environment, the workflow and expected result, and where you are stuck. We review feasibility and agree on scope, acceptance criteria and a quote before work starts.
Ask about paid setup · Discuss sponsorship · leeguoooo@gmail.com
Want to support ongoing maintenance? Email us to discuss a one-time or recurring contribution.
Community bug reports and feature requests remain welcome on GitHub. Please keep credentials and private account data out of inquiries.
MCP calls can choose an installed local instance with instance:"i13" (including
phone_status and input tools); phone_instances lists them without reconnecting.
Omit it to keep the configured daemon. Missing instances fail without selecting a
different phone. See per-call instance routing.
Related MCP server: ios-mcp
Why iphone-use
Any app, no API needed. It works on Health, banking and payment apps (even the ones that black out screen capture), chat apps and your own app under test.
USB or Wi-Fi. Set the phone up once with a cable; after that it can be driven over Wi-Fi through an encrypted tunnel.
Old iPhones too. iOS 15 and later, without updating the phone.
Fast. Its own XCTest device runner reads the screen in about 0.1 s. A tap that returns the settled screen takes about 1.9 s (numbers).
Honest results. Every action reports whether it was applied, not sent, or unknown, and whether a retry is safe. Taps on covered elements are refused, not passed off as done.
Replayable flows. A task done once becomes a flow that replays in one call, with no model and no tokens.
Remote control for people, too. Drive the phone yourself from a browser or another iPhone, on the LAN or from anywhere: a live H.264 screen, mouse and keyboard, and pairing by QR code. Agents and people share one phone without stepping on each other (details).
Watch the agent live. The phone's screen shows in the agent host's side panel (where MCP Apps is supported), in a browser, or in the iOS app.
Nothing on the Mac moves. It never touches the Mac's screen, cursor or focus.
What you can ask
Open Health and tell me how many steps I walked each day this week.
Find the newest message from Mom in WeChat and read it to me.
Collect every transaction on this bank statement page, all the way to the end.
Write this 2,000-word draft into a new note in Notes.
Run my app's login test on the real phone and tell me which step fails.The agent finds an app by its name (launch_app {"app": "Health"}) and types long text (up to 20,000 characters) in one step. It can read a list across pages and say whether it reached the end.
How it works
Claude Code / Codex / any MCP client ──MCP──┐
scripts and other agents ─────────────HTTP──┤
▼
iphone-use daemon on your Mac ──USB or Wi-Fi──▶ XCTest runner on the iPhone
▲
you, in a browser or the iOS app ───────────┘One daemon per phone runs on the Mac. It reads the screen as an accessibility tree (text, not pixels), sends taps and typing through a small XCTest runner it installs on the phone, and checks that the screen has settled before it answers. Agents and people go through the same daemon, and one of them holds the phone at a time.
Install
You need
A Mac with macOS 15 or later, and Xcode signed in to an Apple ID (a free one works).
An iPhone with iOS 15 or later and Developer Mode on (Settings → Privacy & Security → Developer Mode).
A USB cable for the first setup. Plug the iPhone in and unlock it before you install.
The installer then sets the phone up and shows an agent opening Settings and reading it. It registers the MCP server with Claude Code and Codex, and with Cursor and Claude Desktop when they are installed (IPHONE_USE_NO_CLIENT_SETUP=1 skips those two), and opens the control page in your browser with a one-time sign-in, so there is no password to copy. If anything is missing, it says what and how to fix it.
iphone-use status # is the phone ready for agents?
iphone-use doctor # what is missing, and how to fix it
iphone-use setup # set the phone up again (after a new iPhone or Xcode)
iphone-use try # a harmless check: open Settings, read the screen, go Home
iphone-use login # sign a browser in again, with a QR code for the iPhone
iphone-use upgrade # update everything
~/.iphone-use/uninstall.sh --dry-run # what uninstalling would remove (drop --dry-run to do it)Or ask your agent to install it
Paste this into Codex or Claude Code on the Mac the iPhone is plugged into:
Install iphone-use (https://github.com/leeguooooo/iphone-use) on this Mac so you can drive my iPhone.
1. Run: curl -fsSL https://raw.githubusercontent.com/leeguooooo/iphone-use/main/install.sh | sh
2. Run `iphone-use doctor`. Fix what it reports that can be fixed from the terminal, and run
`iphone-use setup` when it tells you to.
3. Some steps only I can do: signing in to an Apple ID in Xcode, tapping Trust on the iPhone,
turning on Developer Mode, unlocking the phone. For those, stop, tell me exactly what to do,
and wait until I say it is done. Never ask me for a password or a code.
4. Run `iphone-use doctor` again until nothing is missing, then `iphone-use try`.
5. If the phone_* tools are not loaded in this chat yet, tell me to start a new chat.Codex and Claude Code plugins
Each adds the skill and the MCP server; they still need the install above:
codex plugin marketplace add leeguooooo/iphone-use && codex plugin add iphone-use@iphone-useIn Claude Code:
/plugin marketplace add leeguooooo/iphone-use
/plugin install iphone-use@iphone-useRemote control for people
The same daemon that serves agents lets you drive the phone yourself:
From a browser. The live screen arrives as H.264 encoded on the phone: native resolution at about 50 fps on the LAN, and a lighter mode for slow links. Click to tap, drag to swipe, hold to long-press, scroll with the wheel or trackpad. Type straight into the phone with your input method, and ⌘V pastes. The Controls panel lists the screen's elements so you can tap by label, and the Flow panel records what you do as a replayable flow. The page follows your browser's language (English or Chinese), with a switch in the sidebar.
From another iPhone. The native iOS app (minimum deployment target iOS 15) pairs by scanning a QR code on the Mac's page, with no address or password to type. It plays the screen with the hardware decoder and switches between your phones. Install it with
iphone-use setup-app, which signs it with your own team; with a paid team the app also controls its own phone with no Mac (details).Screens hidden from capture. Banking and payment apps black out screen capture; there you see a live wireframe of the screen's elements instead of a blank picture, and you can still tap through it.
From anywhere. Reach the Mac through an authenticated HTTPS reverse proxy or a VPN such as Tailscale, and control the phone from outside your network (security notes).
No collisions. A phone has one owner at a time. While you are in control, an agent waits or is told who holds the phone, and the reverse is true too.
Nothing on the Mac moves while you do it: no window takes focus and the cursor stays put.
Use it from an agent
To save a screenshot on the phone for use in another app, call
phone_shortcut {"name":"save_screenshot"}. It requests an iOS system capture
in Photos; check the new image before claiming it saved. phone_screenshot
returns an image to the caller. Saving requires an updated native Runner.
A daemon on your Mac drives the phone and offers the same control in three forms:
an MCP server for Claude Code, Codex, Claude Desktop, Cursor and other MCP clients;
an HTTP API for agents and scripts (
/agent/*);a web page and a native iOS app for people (remote control).
The installer registers the MCP server with Claude Code, Codex, Cursor and Claude Desktop when they are present (IPHONE_USE_NO_CLIENT_SETUP=1 leaves Cursor and Claude Desktop alone). Any other MCP client needs only the command; on this Mac the server finds the daemon and its token by itself:
{
"mcpServers": {
"iphone-use": {
"command": "/Users/YOU/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp"
}
}
}Use the absolute path (replace YOU) and merge the entry into the file; keep the servers already there.
Cursor:
~/.cursor/mcp.json(or.cursor/mcp.jsonin a project), themcpServersblock above.Claude Desktop:
~/Library/Application Support/Claude/claude_desktop_config.json, themcpServersblock above; quit and reopen the app.Gemini CLI:
~/.gemini/settings.json(or.gemini/settings.jsonin a project), themcpServersblock above.Windsurf / Devin Desktop:
~/.codeium/windsurf/mcp_config.json(newer builds:~/.config/devin/mcp_config.json), themcpServersblock above.VS Code / Copilot:
.vscode/mcp.jsonin a workspace, or MCP: Open User Configuration for every workspace. The key isservers:{ "servers": { "iphone-use": { "type": "stdio", "command": "/Users/YOU/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp" } } }opencode:
~/.config/opencode/opencode.json(oropencode.jsonin a project). The key ismcpand the command is an array:{ "$schema": "https://opencode.ai/config.json", "mcp": { "iphone-use": { "type": "local", "command": ["/Users/YOU/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp"], "enabled": true } } }
Or call the HTTP API directly:
AUTH="Authorization: Bearer $TOKEN"; CTL="X-Phone-Control: 1"
curl -s -H "$AUTH" $HOST/agent/elements # the screen as text + a snapshot token
curl -s -H "$AUTH" -H "$CTL" -X POST "$HOST/agent/input?return=delta" \
-d '{"type":"tap","element":7,"snapshot":"…"}' # tap, then what changed (~2 s)For app developers and testers the same tools also read the phone's logs and crash reports, record the screen to MP4, install a build (.ipa, .app or an https link), set a simulated location, and keep per-app notes that agents read when they enter the app. IPHONE_USE_MCP_TOOLS=readonly exposes only the tools that read, and iphone-use-mcp --listen 127.0.0.1:PORT serves MCP over HTTP. A loopback listener is reachable only from the same Mac; for a client on another machine, put it behind an authenticated tunnel, or listen on another address with IPHONE_USE_MCP_TOKEN set (it is required off loopback).
No phone at hand? While you develop, drive an iOS Simulator the same way (preview, needs Xcode): iphone-use simulator start "iPhone 16 Pro" builds and starts everything, then point the agent at it with claude mcp add iphone-sim -e IPHONE_USE_INSTANCE=sim -- iphone-use-mcp. See Simulators.
See the MCP tools and the Agent API reference.
Safety and privacy
Secrets stay with you. For logins, passcodes and Face ID the agent stops and asks you to finish on the phone. It never asks for the password or the code.
Limits the agent cannot talk its way past. The daemon refuses apps you block (
IPHONE_USE_BLOCKED_APPS, for exampleWallet, Passwords) and caps the actions per task (IPHONE_USE_MAX_ACTIONS), whatever the agent's prompt says (details).You see what it does. The web page shows each action over the live screen in plain words, for example
Tapped “Wi-Fi”. Typed text is never kept, only its length.Local by default. The daemon listens on
127.0.0.1:44321behind a password, and the runner on the phone accepts only requests signed with a per-launch token. The iOS 15/16 LAN path is unencrypted (security).No telemetry. This build sends no usage data (telemetry · privacy policy).
phone_home_layout reads the paired phone's persisted home pages, Dock,
folders and widget stacks without opening folders or taking screenshots.
Use page to limit the response. Its page indexes include hidden/Focus-filtered
pages; use fresh UI elements before dragging, rather than treating grid slots
as screen coordinates.
Flows: do a task once, replay it with no model
Tasks you repeat are kept as flows: reviewed per-app scripts that replay without a model, so they cost no tokens. The official registry is leeguooooo/iphone-use-flows. Record a flow in the browser's Flow panel and publish it there with a PR.
Agents find flows on their own: when one enters an app, the response lists that app's flows. After an agent does a multi-step task in an app that has none, it asks whether to save the task as a flow; the daemon has already recorded the steps as a draft. The daemon keeps the registry fresh by itself (IPHONE_USE_FLOWS_NO_AUTO_UPDATE=1 turns that off, IPHONE_USE_FLOWS_NO_SUGGEST=1 silences the save suggestions).
MCP=~/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp
$MCP flow update # sync the registry (sha256-checked)
$MCP flow list # what is available
$MCP flow run health/export-all-zh-cn # replay one; failures say which step and why
$MCP flow draft --out my-task.json # what you just did on the phone, as a draft flowFlows also make rerunnable test suites ($MCP test suite.yaml, with an exit code ready for CI) and scheduled runs the daemon starts on a cron line ($MCP schedule add): see Testing and scheduled runs.
Speed
iphone-use drives the phone through its own XCTest runner (runner/). It speaks the WebDriverAgent HTTP API, and it is built for speed:
It reads the screen with one accessibility snapshot of a fixed attribute set, instead of walking the element tree attribute by attribute.
It synthesizes touches directly and does not wait for the app to go idle before acting; the daemon checks afterwards that the screen has settled.
It is one small app with a built-in HTTP server and MJPEG stream; there are no extra dependencies on the phone.
Measured on the same iPhone 13 over USB, against agent-device (October 2026):
iphone-use runner | agent-device | |
Read the screen (accessibility tree) | 0.08–0.135 s | ~0.7 s |
Tap, then wait for the screen to settle | 1.0 s | 2.8–3.3 s |
Through the full agent API, a tap by label that returns the settled change takes about 1.9 s on an iPhone 17 Pro Max; the WebDriverAgent-based release took 4.2 s. The live view streams at 27–28 fps. Each phone gets its own daemon and runner, so two phones driven at the same time run as fast as either one alone.
More
Full guide: browser and iOS app, flows and the flow registry, Wi-Fi and iOS 15/16, lifecycle, configuration, security, development.
Agent API reference · MCP tools · Architecture · Device setup pitfalls · Device runner · Privacy
Product guide: installation, MCP setup, flows and how it compares.
Issues and ideas: GitHub issues.
Related MCP servers
iPhone MCP: another macOS MCP server for real iPhones and simulators.
ios-mcp: iPhone and iOS Simulator automation through an observe/act/verify loop.
iphone-use focuses on a real device runner, explicit action outcomes and deterministic flow replay. These are independent alternatives, not dependencies or endorsements.
License
PolyForm Shield 1.0.0. This project is source-available. Non-competing use, modification and distribution are permitted subject to the license; using the software to provide a competing product or service, including a free one, is prohibited. See licensing scope and third-party notices.
The *-use family
Small, composable CLIs that give an AI agent hands on one real thing. Same shape
everywhere: curl … install.sh | sh to install, JSON on stdout, and a companion
skill that teaches your agent to drive it — each repo's own README says how to
install its skill, since the right way differs per project and per release.
Repo | Gives your agent |
A real browser — logged-in sessions, forms, scraping, screenshots | |
Email — read, search, send, triage across Gmail / QQ / 163 / any IMAP | |
WeChat on macOS — send messages, query contacts and history | |
Discord — messages, channels, forums, webhooks (REST-only, Rust) | |
Many logged-in accounts per site — capture, switch, apply sessions | |
Your personal profile, safely — fill signup / KYC / checkout forms | |
Bitwarden / Vaultwarden — headless passkey (FIDO2) login | |
Your ChatGPT subscription as a coding-agent backend — no API key | |
The macOS desktop itself | |
Read-only PixCake probing — snapshot / diff / SQLite inspection |
Author
Built by 郭立 (Guo Li / leeguoo) — leeguoo.com · GitHub · X · more tools in the *-use family.
Available Tools
41 toolsphone_app_notesADestructiveIdempotent
Read or replace persistent notes for one bundle id about app navigation and quirks. Resolve bundle with phone_apps; without text, read the existing Markdown. With text, replace it entirely (not append), at most 16 KiB UTF-8; read first to preserve useful notes. An empty text clears the notes. The first screen read in an app also includes them in its registry block. Use phone_flow_draft for executable task replay, not these notes. Writes affect future sessions; never store passwords, codes or personal data. Errors do not prove a write was delivered: read back before retrying.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | Markdown that REPLACES the app's notes (at most 16 KB). Omit to read them. | |
| bundle | Yes | The app's bundle id (phone_apps finds it by name). | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare destructive/idempotent/not-read-only; the description adds the crucial mechanics: replacement rather than append, the 16 KiB UTF-8 cap, that empty text wipes notes, that writes persist into future sessions, a no-secrets safety rule, and that errors don't prove delivery so a read-back is required before retrying. This is materially more than the annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the read/replace split in the first sentence, then the operational details in descending order of importance (destructive semantics, size, clearing, sibling routing, safety, error recovery). Every sentence carries distinct information; nothing is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, no-output-schema tool, the description covers the write model, capacity limit, clearing behavior, persistence across sessions, safety constraints, retry semantics, and how notes surface in the screen registry block. Nothing an agent needs to call it safely is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds semantics the schema only implies: the size limit in UTF-8, that empty string is a distinct clearing operation versus omission meaning read, and that the bundle maps to an entry found by phone_apps. The instance parameter is left entirely to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb pair and resource: 'Read or replace persistent notes for one bundle id about app navigation and quirks.' It also scopes the content type (navigation and quirks) and names the sibling it is not (phone_flow_draft), so an agent can distinguish it from the flow-replay tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit branches: 'without text, read', 'with text, replace it entirely', 'an empty text clears the notes', plus 'read first to preserve useful notes' and 'Use phone_flow_draft for executable task replay, not these notes.' It also routes bundle resolution to phone_apps, which are all the when/how decisions an agent must make.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_appsARead-only
Find an app's bundle id by name instead of guessing it. Read-only: it never touches the screen and takes no owner lease. Searches the phone's own app list, then a bundled catalog of popular apps (bundle ids verified against Apple's lookup API, plus Apple's built-in apps), then the App Store only when neither matched. Exact name/alias matches hide partial ones. Each candidate carries name, bundle_id, source (installed|catalog|apple), installed_verified, and publisher when known; installed_verified=false with installation_checked=false means unknown, not absent. Pick the candidate whose name and publisher fit, then launch it with launch_app bundle — or give launch_app app the exact name and let the daemon resolve it (it refuses ambiguous names with candidates instead of guessing).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | 1–30 candidates. Default 10. | |
| query | Yes | The app's name as a person would say it (微信, WeChat, 招商银行, Settings) or a bundle id. 1–100 characters. | |
| source | No | `auto` (default: the phone's own app list, then the bundled catalog, then the App Store), `installed`, `catalog` (offline), or `apple`. | |
| country | No | App Store storefront for the store search, two letters. Default: the store the phone's apps came from (or IPHONE_USE_STORE_COUNTRY, else the Region setting); `cn` only when none is known, with a warning. The answer's `country_source` says which. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint/openWorldHint annotations: no screen interaction, no owner lease, ordered source fallback with App Store last, exact matches suppressing partials, and the important caveat that installed_verified=false with installation_checked=false means unknown rather than absent. It also discloses the ambiguity-refusal behavior instead of guessing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and safety statement are front-loaded, and the candidate-field/next-step guidance follows logically. It is dense — long semicolon-linked clauses in the final sentence — but nearly every clause carries decision-relevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter, read-only lookup with no output schema, the description covers what each candidate field means, what the edge-case flags imply, and how to act on results. There is no output schema gap left unaddressed and no missing prerequisite.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so query/limit/source/country/instance are already documented at the parameter level. The description reinforces `source` ordering semantics and the name-vs-bundle-id form of `query`, but adds little about limit, country, or instance. Baseline 3 is right when the schema carries the detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource+goal: "Find an app's bundle id by name instead of guessing it." That immediately separates it from phone_install_app (which installs) and from launch tools, so an agent can pick it without reading schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the exact search cascade (phone list -> bundled catalog -> App Store) and what selects each stage via `source`. It also names the follow-on alternatives explicitly: pass a bundle id to launch_app, or hand launch_app the exact name and let the daemon refuse ambiguity rather than guess.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_capabilitiesARead-only
Inspect supported UI operations and cached readiness before choosing an action. Unlike phone_status (device/setup/owner diagnostics), this names the backend's single-step, batch and perform vocabularies, observation support and accepted modes. Read-only: no device connection, wake or owner lease. supported is static; available is cached with blocked_by naming locked, released, reconnecting, human_handoff, owned_by_other or offline. ok:null is unknown, never proof of readiness. This describes UI control only, not app installation or other device management. If blocked, follow phone_status guidance instead of probing with actions.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint annotation: it discloses 'no device connection, wake or owner lease', that 'supported is static' while 'available is cached', the full blocked_by vocabulary, and critically warns 'ok:null is unknown, never proof of readiness'. This is exactly the kind of subtle behavioral context annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded and most sentences carry distinct, non-redundant facts. It is quite dense and packs several caveats into a short span, but nearly every clause (vocabularies, blocked_by values, ok:null warning) earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description explains the return semantics fully: supported vs available, blocked_by enumeration, and the ok:null ambiguity. Combined with the read-only annotation and complete parameter schema, an agent has everything needed to call and interpret it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single 'instance' parameter is fully documented in the schema, so the baseline of 3 applies. The description adds no further meaning to the instance parameter, but it doesn't need to since the schema is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Inspect supported UI operations and cached readiness') and explicitly differentiates from the sibling phone_status ('Unlike phone_status (device/setup/owner diagnostics), this names the backend's single-step, batch and perform vocabularies'). An agent can distinguish the tool from its siblings without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('before choosing an action') and a named alternative with condition ('If blocked, follow phone_status guidance instead of probing with actions'). Also delimits scope ('This describes UI control only, not app installation or other device management'), routing installation work away.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_collect_listA
Read a scrolling list across pages in ONE call instead of scroll + read turns: reads the rows of one kind (default Cell) on screen, swipes one page, reads again, and stops when the exposed page repeats (duplicate_page), end_label is on screen, or max_pages (default 6, max 10) is reached. Rows are deduplicated on (kind, identifier, label, value): different exposed identifiers preserve identical-looking rows. Absent or reused identifiers with identical content may still collapse, so reconcile counts the user cares about. complete is true ONLY when end_label was seen (complete_scope=end_label_visibility); business_complete is always false. duplicate_page is not proof the list ended. Changed pages continue even with no new deduplicated rows. Only what the accessibility tree exposes is collected. It swipes the live list, so the screen is left where the last page was; snapshot is that last page. Optional snapshot (from a read of the target app; never filled in for you): a known foreground app mismatch refuses before any swipe (app_changed; setup may already have affected the device). Results name the app. Owner/privacy changes stop further reads and scrolling. swipes_confirmed counts Native ACKs, swipes_uncertain may have reached the device; never replay uncertain scrolling. Requires phone_status drivable=true.
| Name | Required | Description | Default |
|---|---|---|---|
| region | No | Only rows whose centre lies in this normalized `[x, y, w, h]` area, and swipes inside it (default: the whole screen). | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| row_kind | No | Element kind of one list row (default `Cell`; e.g. `StaticText`, `Button`, `Link`). A row without a label is named by its texts. | |
| snapshot | No | A snapshot from an earlier read of the target app: when given, a different foreground app refuses before any swipe. Not filled in for you. | |
| direction | No | `down` (default) reveals rows below, `up` rows above. | |
| end_label | No | Exact label that marks the end of the list ("没有更多了", "No more results"). Only this makes the result `complete:true`. | |
| max_pages | No | Pages to read, 1–10 (default 6). One swipe between pages. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only say readOnlyHint=false, openWorldHint=true, idempotentHint=false, destructiveHint=false; the description adds far more: stop conditions (duplicate_page, end_label, max_pages), why duplicate_page is not proof of the end, that only accessibility-tree-exposed content is collected, that the screen is left on the last page, that a snapshot mismatch refuses before swiping, that owner/privacy changes halt reads, and that uncertain swipes must never be replayed. This is consistent with readOnlyHint=false since it swipes the live device.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Core purpose is front-loaded in the first clause and nearly every sentence carries operational signal. It is dense to the point of a run-on and repeats the duplicate_page idea ('stops when the exposed page repeats' then 'duplicate_page is not proof the list ended'), which costs some tightness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of explaining returns and does so: rows and deduplication, `complete`, `business_complete`, `duplicate_page`, `snapshot`, `swipes_confirmed` vs `swipes_uncertain`, and that results name the app. For a 7-parameter, state-changing tool, an agent has enough to call it correctly and interpret the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description earns above that by adding semantics the schema lacks — the dedup key (kind, identifier, label, value), that end_label alone drives complete:true, that max_pages caps at 10, and that snapshot is never auto-filled. It does repeat some schema-stated defaults (default Cell, default 6).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — read a scrolling list across pages in one call, reading rows of one kind, swiping, re-reading — and explicitly contrasts it with the 'scroll + read turns' pattern a sibling like phone_scroll would require. An agent can tell what it does without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Says when to prefer it (batch list collection instead of alternating scroll and read turns) and gives a prerequisite, 'Requires phone_status drivable=true'. It does not explicitly name the alternatives (phone_scroll, phone_scroll_find) or state when not to use it, so routing between list tools is still partly inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_crashesARead-only
Crash reports on the phone: without name, list them (name, process, time, size; filter with process); with a name from the list, read that report — exception, crashed thread and backtrace. Read-only: it never touches the screen.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Read this report (a `name` from the list). Omit to list reports. | |
| limit | No | List: at most this many reports (default 50). | |
| process | No | List: only reports of processes whose name contains this text. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already declares the safety profile, and openWorldHint is present, so the bar is lower; the description still adds the notable fact that the tool never touches the screen, which matters among screen-manipulating siblings. It also discloses the returned fields for each mode (name/process/time/size vs exception/crashed thread/backtrace), useful context given there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence pair, front-loaded with the resource and the mode switch, with no redundant restatement of the tool name. Every clause carries either mode selection or return-contract information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so for both modes. The only omissions are the default page size and the `instance` targeting behavior, both of which are covered by the schema, so nothing an agent needs to call correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so a 3 is the floor, but the description goes further by explaining the coupling between `name` and the result shape (a report name switches the tool from list to detail mode) and that `process` is a list-only substring filter. It does not mention `limit` or `instance`, which the schema handles adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (crash reports) and both verbs it implements: list without `name`, read a single report with `name`. The dual-mode split is spelled out so the agent can predict the output shape before calling, and the closing clause separates it from the screen-driving siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The when-to-use logic is explicit for both modes: omit `name` to enumerate, supply a `name` taken from the list to read detail, and use `process` to narrow the list. What's missing is any pointer to an alternative tool for related diagnostics (e.g. logs), so an agent comparing siblings gets context but no explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_elementsARead-only
The current screen as text — use this instead of screenshots (an order of magnitude cheaper). Rows carry kind, label, identifier, value, rect and state, plus a snapshot token for phone_tap_element; indexes and snapshot tokens are valid for this read only. query keeps only rows containing that text (label, identifier, value, placeholder) and limit caps the rows. selector ANDs exact kind/identifier/name/label/value/enabled fields and case-sensitive label_contains; optional selector.index (integer0..199) selects one match after scope/query and before limit, retaining the original snapshot row index. Unknown earlier candidates or incomplete tree scope refuse the ordinal. name matches a non-empty identifier, otherwise the label; value_present checks whether the snapshot supplies a string value (including empty/truncated); false means omitted/null on non-secure rows, not a complete AX null; protected SecureTextField rows match neither boolean. It filters AX rows, not OCR. Queries preserve each row's index and the snapshot, so taps still work. Screen text shows at most 200 rows and 32 KiB of row text; an omission notice means a partial view, not absence or list completion. Narrow with query when needed. include_hidden=true expands off-screen/not-drawn rows instead of summarizing five labels; these rows are marked hidden and must be scrolled into view before acting. On the first read in a newly entered app a registry block lists saved flows for it: if one does the task, phone_flow_run it instead of tapping step by step. An alert block means a system alert is up: answer it with an alert step, taps on alerts do not act. ocr=auto adds text read from pixels when the tree has no interactive rows (games, canvases), ocr=force always; such rows have no element, tap their x/y with phone_tap. value_truncated marks an incomplete preview; use phone_read_value for the bounded full AX value. Requires phone_status drivable=true.
| Name | Required | Description | Default |
|---|---|---|---|
| ocr | No | Text read from a fresh screenshot (Vision OCR) for screens the tree cannot describe (games, canvases, video): `auto` adds it only when the tree has no interactive rows, `force` always, `off` (default) never. | |
| limit | No | Return at most this many rows; the answer says how many were left out. | |
| query | No | Keep only rows whose label, identifier, value or placeholder contains this text (case-insensitive). Rows keep their original index. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| selector | No | Literal case-sensitive field filters combined with AND; preserves original snapshot indexes. Requires literal criteria; optional index0..199 chooses one match in scoped/query tree order before limit. Row index remains the full snapshot index; incomplete or unknown earlier matches refuse an unproven rank. Filters AX elements, not separate OCR text. | |
| include_hidden | No | Show off-screen/not-drawn rows individually in the text, with their original indexes and hidden state, instead of a five-label summary. The 200-row / 32 KiB text budget still applies. Scroll them into view before acting. Default false. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=true, so safety is covered. The description adds substantial context beyond that: snapshot tokens are valid for this read only, 200-row / 32 KiB text budgets, omission notices mean partial views not absence, hidden rows must be scrolled into view before acting, and alert blocks block taps. That is genuinely useful behavioral context, though the return format is described in dense, hard-to-parse prose rather than structured disclosure; a 3 reflects solid added value over the annotation baseline without being exemplary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loading is excellent — the cheapest-vs-screenshot claim and the sibling reference come first — but the remainder is a single dense paragraph mixing return shape, filter semantics, budgeting, alerts, registry, OCR and the value_truncated handoff. Multi-topic content this important (alerts block taps, hidden rows need scrolling) is buried mid-paragraph where an agent scanning for call conditions can miss it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter read tool with no output schema, the description covers return row fields, size/row budgets, truncation semantics, hidden-row behavior, alert handling, registry discovery, and OCR fallbacks — everything needed to call and interpret it. Nothing material is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real semantic layering: it explains the evaluation order (selector.index applies after scope/query and before limit) and that the ordinal retains the original snapshot row index, plus that ordinal matching refuses when earlier candidates are unknown or scope is incomplete. It also clarifies that value_present covers empty/truncated strings and that SecureTextField rows match neither boolean.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a concrete verb+resource: 'The current screen as text', and immediately distinguishes itself from phone_screenshot ('use this instead of screenshots'). It also names the exact return shape (kind, label, identifier, value, rect, state, snapshot), so an agent knows precisely what it gets without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing to alternatives: phone_tap_element consumes the snapshot token, phone_read_value for full bounded values, phone_flow_run when the registry block shows a saved flow, phone_tap with x/y for OCR rows. It also states preconditions ('Requires phone_status drivable=true') and when to narrow ('Narrow with query when needed').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_draftA
Turn what you just did on the phone into a flow. The daemon records every action it applied in the current app (single actions and phone_run_steps batches, not flow runs); this returns that trail as a flow v1 document (snapshot taps become label/identifier locators, typed text becomes named inputs and is never included) plus a todo list of what to fix. Call it when a response carried flow_suggestion, or after any multi-step task with no matching flow — but ASK THE USER before saving. With save_as, the draft is written to that new file and validated; fix the todo items, prove it with phone_flow_run(id=), then publish only with the user's OK.
| Name | Required | Description | Default |
|---|---|---|---|
| save_as | No | Optional path to write the draft flow to (a new `.json` file; existing files are never overwritten). It is validated after writing. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as a non-read-only, non-destructive, non-idempotent write, and the description adds substantial context beyond them: what is recorded (single actions and phone_run_steps batches, not flow runs), what is deliberately excluded (typed text never included), and the file-write safety property (save_as never overwrites existing files, validated after write). This is rich disclosure an agent could not infer from structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded, with the core action stated first and the workflow and caveats following. The parenthetical aside is slightly run-on, but every sentence carries actionable information; minor tightening would make it a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description fills that gap by explaining the return value (a flow v1 document with locator/input mapping plus a `todo` list). Combined with prerequisites and the follow-on workflow, an agent has everything needed to call and use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents both params, making baseline 3 appropriate. The description goes further by explaining the save_as lifecycle (write to new file, validate, then fix/run/publish), adding meaning beyond the schema, though it does not discuss the `instance` parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('turn what you just did on the phone into a flow') and clarifies the mechanism (daemon action trail, not flow runs). It clearly distinguishes this drafting tool from siblings like phone_flow_run, phone_flow_list, and phone_flow_publish.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit triggers ('when a response carried flow_suggestion, or after any multi-step task with no matching flow'), an exclusion, and a mandatory prerequisite ('ASK THE USER before saving'). It also names the follow-on workflow (fix todo, phone_flow_run, publish with user OK), routing the agent precisely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_infoARead-only
Inspect an installed flow before phone_flow_run: returns metadata, declared inputs and step templates with placeholders, never runtime input values. Use an id from phone_flow_list, such as health/export-all: exactly two lowercase app/flow slugs, not a local file path. Check app, locale, verification and side effects before execution. Invalid or missing ids return an error without running or downloading anything; use phone_flow_update if the registry store is empty. To inspect a new local draft, validate its file instead of guessing a registry id.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Registry id such as `health/export-all`. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description discloses that invalid or missing ids return an error 'without running or downloading anything', that placeholders are shown but runtime values are not, and that the caller should verify app, locale, verification and side effects before execution. That is meaningful behavioral context an agent cannot get from annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose and return contents, then the id constraint, then failure behavior and siblings. It is dense with multiple clauses per sentence, but virtually every clause carries actionable information, so little is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the gap left by the absent output schema by describing what is returned (metadata, declared inputs, step templates with placeholders) and what is not. Format constraints, error behavior, and workflow position are all present, so an agent has enough to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are already documented. The description adds real value beyond the schema by specifying the id format ('exactly two lowercase app/flow slugs, not a local file path') and giving a concrete example, though it says nothing extra about the instance parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Inspect an installed flow') and immediately enumerates what it returns ('metadata, declared inputs and step templates') plus what it explicitly does not return ('never runtime input values'). It is clearly distinguished from siblings phone_flow_run (which executes), phone_flow_list (which enumerates), and phone_flow_update (which populates the registry).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly prescribes the precondition ('Use an id from phone_flow_list'), names alternatives with their conditions ('use phone_flow_update if the registry store is empty', 'validate its file instead of guessing a registry id' for local drafts). It also states the purpose in the workflow ('before phone_flow_run').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_listARead-only
Find installed deterministic per-app flows before exploring a task: a matching phone_flow_run can replay it without model decisions or screenshots. Filters category, app and verified combine; app is a registry app directory or bundle id, not a display-name search. Compact rows report id, description, inputs, risk and compatibility with this phone's app version; detail=true returns full JSON. verified means a recorded hardware run, not a guarantee for every current version. Use phone_flow_info to inspect effects before running. An empty store needs phone_flow_update once; a filter with no matches is not proof that the whole store is empty.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | Only flows for this app directory (e.g. `health`) or bundle id. | |
| detail | No | Return every field of every entry (verified_on, app versions, step counts) as JSON text instead of one compact row per flow. | |
| category | No | Only flows in this registry category (e.g. `health`, `system`, `finance`, `im`). | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| verified | No | Only flows with at least one recorded hardware verification. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint/openWorldHint annotations: it discloses that filters AND-combine, that app takes a registry directory or bundle id and is explicitly NOT a display-name search, what the compact row contains, what detail=true changes, and the important caveat that 'verified' means a recorded hardware run rather than a per-version guarantee. That is exactly the kind of semantic and trust caveat structured fields cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose then alternative then caveats, and every clause is substantive rather than filler. It is dense to the point of being a run-on in places, but trimming any sentence would lose real information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by describing the compact row fields (id, description, inputs, risk, compatibility) and what detail=true returns. Combined with the empty-store and verification caveats, an agent has enough to call this correctly and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description earns a bump by adding filter-combination semantics and the app-parameter disambiguation ('registry app directory or bundle id, not a display-name search') that the schema text does not state. It does not add much on `instance` or `detail` beyond what the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Find installed') and resource ('deterministic per-app flows') with scope ('before exploring a task') and immediately distinguishes itself from phone_flow_run by noting a match can replay without model decisions. An agent can tell this apart from phone_flow_info, phone_flow_run and phone_flow_update without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly positions the tool in a workflow ('before exploring a task'), names the alternative for deeper inspection ('Use phone_flow_info to inspect effects before running'), and gives the remediation path for the empty case ('An empty store needs phone_flow_update once'). It even forecloses a false conclusion ('a filter with no matches is not proof that the whole store is empty').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_publishADestructive
Publish a working flow to the official registry through the user's GitHub CLI login: forks if needed, writes the flow and any new app.json, rebuilds index.json, pushes a branch and opens a public pull request. Requires confirm=true only after the user agrees to that publication. First create a draft with phone_flow_draft and prove it with phone_flow_run; use phone_flow_report to report an existing flow problem. Put device/iOS/date in verified_on; unverified flows open a draft PR. source is a local file or installed flow id; aliases are foreground-app labels by language. Returns id, repo, branch, head and pr_url; inspect the URL before retrying a failed publication.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Registry id to publish as, `<app>/<flow>` lowercase slugs. | |
| note | No | What was verified, where, and anything a reviewer should know. | |
| source | Yes | Flow file path, or an id installed with `flow add`. | |
| aliases | No | Foreground-app labels per language (e.g. ["Health","健康"]). | |
| confirm | No | Must be true; set only after the user agreed to open a public PR. | |
| app_name | No | Human app name, used only when the app is new to the registry. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, openWorldHint=true, and non-idempotence, so the safety bar is partly covered. The description adds valuable context beyond that: it forks if needed, writes app.json, rebuilds index.json, pushes a branch, opens a public PR, opens a draft PR for unverified flows, and advises inspecting pr_url before retrying. This is exactly the extra behavioral detail an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and packs a lot of useful detail into one paragraph. It is dense but every sentence mostly earns its place; minor deductions for being a single run-on paragraph and for the wasted mention of verified_on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex, non-idempotent mutation tool with no output schema, the description covers the return fields (id, repo, branch, head, pr_url), the confirmation gate, and retry guidance. It is nearly complete, but the phantom verified_on parameter and thin coverage of instance/app_name/note leave small gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds a little meaning for source ('local file or installed flow id') and aliases ('foreground-app labels by language'), but it also references a 'verified_on' field that is not present in the schema (which has additionalProperties=false), creating a potential misdirection for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Publish a working flow to the official registry') plus the mechanism ('through the user's GitHub CLI login'). It clearly distinguishes itself from sibling tools like phone_flow_draft, phone_flow_run, and phone_flow_report by naming them as prerequisites or alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit sequencing: create a draft with phone_flow_draft, prove it with phone_flow_run, and use phone_flow_report for existing problems. It also gives a clear gating condition for confirm=true ('only after the user agrees to that publication'), leaving little ambiguity about when to call it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_reportADestructive
File an issue on the official flow registry for an installed flow that failed (label changed, app updated, wrong locale). Uses the failure captured by the last phone_flow_run of that id — failed step, redacted daemon result, daemon version — so you only add a short note. Screen labels, typed text, and element lists are stripped. OUTWARD-FACING: requires confirm=true after the user agreed. Do not file for a phone that was simply locked/offline, or for a flow you ran on the wrong app/locale.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Registry id of the flow that failed. | |
| note | No | One or two sentences: what you expected, what the phone showed. | |
| confirm | No | Must be true; set only after the user agreed to open a public issue. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and openWorldHint=true, but the description adds material context the annotations cannot: the target is a public, outward-facing registry, an explicit user-consent gate is required, and the submitted payload has screen labels, typed text, and element lists stripped. Not stated is duplicate-issue handling or whether the call is rate-limited, so this falls just short of fully complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose comes first, then the implicit data source, then the privacy guarantee, then the consent gate, then the exclusions — a deliberate ordering with no filler. Every clause (including the failure-type parenthetical) carries decision-relevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter, single-object tool with no output schema and annotations already covering the risk profile, the description supplies the remaining decision inputs: precondition of a prior run, consent requirement, redaction scope, and negative cases. An agent can determine whether to call it and how to prepare the note without further information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3; the description earns an extra point by explaining that the id scopes the report to the last phone_flow_run of that flow (so the failure data is implicit) and that this means the caller 'only add[s] a short note'. The confirm semantics are reinforced as a consent gate rather than a technical flag.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — 'File an issue on the official flow registry' — for 'an installed flow that failed', and the parenthetical enumerates the failure classes (label changed, app updated, wrong locale). This is clearly distinct from siblings like phone_flow_publish, phone_flow_draft, and phone_flow_run without needing their schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives the positive precondition (a failure captured by the last phone_flow_run of that id), the gating requirement (confirm=true only after the user agreed), and explicit exclusions (do not file for a locked/offline phone, or for a flow run on the wrong app/locale). Both when-to and when-not are stated, plus the alternative interpretation of the failure.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_runADestructive
Run one flow exactly once — an installed registry id, or a flow file path (*.json, e.g. a phone_flow_draft you are proving before publishing). A flow that declares outputs returns what it read as top-level outputs (e.g. today's steps); verify=true also checks their shape against the recorded fixture. A read_only/navigation flow that misses an element runs its other-language variant once (fallback_from says so): the daemon validates the whole sequence, holds one control lock, and stops at the first failed step. The happy path costs one tool call and zero screenshots. Requires phone_status drivable=true. Pass the flow's declared inputs; a flow declared risk=side_effect is refused unless confirm=true; a flow whose compat is broken or incompatible for the installed app version is refused unless force=true. The result reports completed/applied counts and the failed step; retry_safe=false means DO NOT replay — inspect phone_elements, repair the flow, do not guess. Unverified flows (verified=false in phone_flow_list) may need a checkpoint screenshot afterwards.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Registry id such as `health/export-all` (see phone_flow_list), or the path of a flow file (`*.json`) — e.g. a draft saved by phone_flow_draft, to prove it on the phone before publishing. | |
| force | No | Run even when compat is `broken` or `incompatible` for this phone. | |
| inputs | No | Runtime string inputs declared by the flow. Values are used for this run only and never persisted. Never pass passwords, codes, or private message content. | |
| verify | No | Compare what the flow read (its `outputs`) against the recorded result shape; `verify.ok=false` names what drifted. | |
| confirm | No | Required for flows declared `risk: side_effect`. Set true only after the user confirmed the target and inputs. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| write_fixture | No | With verify: record this run's output shape as the fixture (types only). Only after checking the values are right. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare a destructive, non-idempotent, open-world write, and the description goes well beyond them: one control lock is held, the daemon validates the whole sequence and stops at the first failed step, retry_safe=false forbids replay, and read_only flows auto-run their other-language variant with fallback_from reported. None of this contradicts the annotations and much of it cannot be inferred from them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and scope, and every sentence carries operational information (return shape, locking, fallback, refusal gates). It is dense as a single unbroken paragraph, which makes the retry_safe=false warning easy to miss, but there is little waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by explaining what comes back: top-level outputs, completed/applied counts, the failed step, verify.ok drift reporting, and fallback_from. For a 7-parameter destructive tool with nested inputs, an agent has everything needed to call it and to interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so per-field meaning is already documented and baseline is 3. The description adds cross-parameter semantics the schema does not: inputs are the flow's declared inputs used for one run only, confirm is gated on prior user confirmation, and force is tied to compat state — a modest but real value add.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Run one flow exactly once') and immediately broadens the id semantics to registry id or *.json draft path, which separates it from phone_run_steps, phone_run_start and phone_flow_publish. It also names the sibling tools an agent would otherwise confuse with it (phone_flow_draft, phone_flow_list, phone_elements).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit preconditions ('Requires phone_status drivable=true'), refusal conditions (risk=side_effect needs confirm, broken/incompatible compat needs force), and an alternative action on failure ('inspect phone_elements, repair the flow, do not guess'). It also states that unverified flows may need a checkpoint screenshot, which routes the agent to the next tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_flow_updateAIdempotent
Mirror the official iphone-use flow registry (github.com/leeguooooo/iphone-use-flows) into the local store. Every file is checksum-verified and strictly validated before it is written; locally added flows are kept. Call once when phone_flow_list reports an empty store, or to pick up new flows. Network only; the phone is not touched.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotent, non-destructive, open-world behavior; the description adds the concrete reason ('locally added flows are kept') plus checksum verification and strict pre-write validation, so an agent knows local edits survive. It does not say what happens to partially failed syncs or whether network errors are reported as a list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with zero padding: what it does, its safety properties, then the call condition. The scope statement is front-loaded and the guidance is last, which is the natural reading order.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the essentials of a network-fetching, store-mutating tool with no output schema: what is written, how it is validated, what is preserved, and that the phone itself is untouched. Missing only failure-mode detail (what an agent sees on a partial or checksum-failed sync).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single optional 'instance' parameter is documented there with its fallback behavior, so the description adds no parameter meaning. Baseline 3 applies when the schema carries the full burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: mirror the official iphone-use flow registry into the local store. It is clearly distinguishable from phone_flow_list (read), phone_flow_draft (authoring) and phone_flow_publish (pushing out) named among the siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit trigger ('Call once when phone_flow_list reports an empty store, or to pick up new flows'), tying usage to a named sibling's output. It also bounds the call ('call once'), which prevents a polling loop.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_holdAIdempotent
Keep the phone for a bounded human-in-the-loop pause inside the current user-requested task (the operator types a PIN, approves a prompt, fetches a code): the idle watchdog will not release it for secs seconds (0 clears the hold; max 14400). When the person signs in (passcode, Face ID, a code), pass private=true: the screen is not read or streamed until you clear the hold; pass private=true again when extending it, since a hold without it lifts the privacy. Not for initialization, health checks, or keeping the phone ready; clear it when the step is done. Fails with device_release_in_progress if the daemon is already releasing, and with phone_owned if another session holds the phone lease.
| Name | Required | Description | Default |
|---|---|---|---|
| secs | Yes | Seconds to keep the phone (0 clears the hold; at most 14400). | |
| private | No | The person is signing in (passcode, Face ID, a code): nothing reads the screen until the hold ends or is cleared — screenshots, elements and live views are refused with `screen_private`. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations: it discloses watchdog semantics, the 0-clears and max-14400 bounds, the private-mode consequence (screen not read or streamed, and that extending without private lifts privacy), and two named failure modes (device_release_in_progress, phone_owned). Annotations only cover the safety profile (readOnlyHint=false, idempotentHint=true), so this added context is substantial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action and bounds, then packs guidance, privacy rules, and error conditions in that order. It is dense but every clause carries operational meaning; the single long paragraph and parenthetical asides make it slightly harder to scan than a bulleted form would.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating, timer-based lease tool with no output schema, the description covers purpose, bounds, privacy implications, cleanup, and failure modes. An agent has everything needed to invoke it correctly and avoid the common misuse of leaving a stale hold.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description earns more by tying `private` to a real situation ('when the person signs in') and warning that it must be re-passed on extension or the privacy lapses — a non-obvious coupling the schema only partly conveys. It does not clarify the `instance` fallback behavior, which the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Keep the phone for a bounded human-in-the-loop pause inside the current user-requested task,' and gives concrete examples of what the operator does (types a PIN, approves a prompt, fetches a code). It also carves out the scope against sibling behaviors ('Not for initialization, health checks, or keeping the phone ready'), so an agent can distinguish it from phone_status or phone_capabilities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use (a human step in the current task), explicit when-not ('Not for initialization, health checks, or keeping the phone ready'), and an explicit termination rule ('clear it when the step is done'). The guidance is actionable without opening any schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_home_layoutADestructive
Read all home-screen pages, Dock, folder pages and widget stacks from the paired iPhone without opening folders, swiping, or taking screenshots. Optional page selects one zero-based page to reduce output. Requires a configured physical target reachable through usbmuxd. This metadata read does not require drivable=true or a running XCTest Runner; never reconnect or open Home solely for this read. Page indexes refer to persisted icon state, including hidden/Focus-filtered pages; they are not the visible page-control indexes. Zero-based row/col positions are derived from icon-state order and grid metrics; null positions or positions_complete=false mean unknown, never safe drag coordinates. Contains no pixel coordinates. An unreadable device returns an error rather than an empty home screen. Writing (action): backup saves the layout as a plist on the Mac; preview compares a plist with the phone by app identifier (added, removed, moved, widgets that would be lost); apply/restore replace the whole layout, only with confirm=true after the person agreed to the preview, and back up the replaced layout first. After a write read again: SpringBoard may bring App Library apps back over a few passes. Undo with action=restore.
| Name | Required | Description | Default |
|---|---|---|---|
| file | No | Absolute path of a layout .plist on the Mac (from a backup). | |
| page | No | ||
| action | No | Omit to read. `backup` saves the current layout on the Mac; `preview` shows what writing `file` would change, by app identifier; `apply` writes `file`; `restore` writes the newest automatic backup (or `file`). | |
| confirm | No | apply/restore only: true writes. Without it the preview comes back and nothing changes; show it to the person and ask first. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations: apply/restore replace the whole layout, the replaced layout is backed up first, and SpringBoard may reintroduce App Library apps across several passes so a re-read is required after writing. Also discloses that an unreadable device errors rather than returning an empty screen. The read/write split is consistent with readOnlyHint=false and destructiveHint=true; a 5 is withheld only because eventual-consistency timing and partial-failure reporting on apply are not pinned down.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded correctly with the read purpose and page-index caveat, but it is a single dense paragraph that folds the read contract, coordinate semantics, device prerequisites, and four write actions into one block with no separation between read and write modes. Every sentence carries information, yet the structure forces re-parsing to find the write rules.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a five-parameter tool with no output schema and a destructive write path, the description covers prerequisites (usbmuxd, no Runner needed), failure behavior (error not empty result), output-field semantics (positions_complete, null positions), write ordering and confirmation, and undo. An agent has enough to invoke both the read and the write modes correctly without guessing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning the schema cannot: page indexes are persisted icon state including hidden and Focus-filtered pages, not the visible page-control indexes, and row/col values are derived from icon-state order and grid metrics with null or positions_complete=false meaning unknown rather than safe. Also clarifies file is a plist on the Mac produced by a backup and that confirm is a human-agreement gate, so the write contract is unambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource set: 'Read all home-screen pages, Dock, folder pages and widget stacks from the paired iPhone', and explicitly separates itself from screenshot/tap siblings ('without opening folders, swiping, or taking screenshots'). The tool is really two modes (read plus backup/preview/apply/restore) bundled under one name, so an agent must read the whole paragraph to learn the tool also mutates, which keeps it short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use and when-not-to-use conditions: no drivable=true or XCTest Runner needed, and 'never reconnect or open Home solely for this read'. For the write path it names the required sequence (backup or preview first, then apply/restore only with confirm=true after the person agreed to the preview) and the recovery path ('Undo with action=restore'). Alternatives are named rather than implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_install_appADestructive
Install an app build on the phone: path of an .ipa or .app on the Mac running the daemon, or an https url of an .ipa (up to 2 GB). fresh=true uninstalls the app first, deleting its data. Apps the operator blocked are refused. Answers with the installed bundle id; it can take minutes. Install only what the user asked for.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | An https URL of an `.ipa` (at most 2 GB); the daemon downloads it. | |
| path | No | Absolute path of an `.ipa` or `.app` on the Mac running the daemon. | |
| fresh | No | Uninstall the app first (its data goes with it), for a clean start. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations: it discloses that fresh=true uninstalls first and destroys app data, that blocked apps are refused, that the call can take minutes, and what is returned (the installed bundle id). That is exactly the extra context an agent needs for a destructive, slow, non-idempotent operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense but front-loaded paragraph: the core action and accepted sources come first, then behavior, then the safety caveat. No filler sentences and nothing redundant with the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-output-schema, four-parameter mutation tool this covers duration, failure mode (blocked apps), data loss, and return value. It omits any mention of the `instance` selector, which is the only remaining gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, but the description adds meaning by presenting path and url as mutually exclusive source choices and by spelling out the side effect of fresh (data deletion). The `instance` parameter is left to the schema, keeping it just short of a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Install an app build on the phone") and enumerates the two source forms (local path vs https url), so an agent immediately understands the operation and its inputs. No sibling tool overlaps with installing apps, and the description makes that distinction obvious without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a real usage constraint ("Install only what the user asked for") plus the conditional rule for fresh=true and the refusal of operator-blocked apps. It stops short of naming alternatives or when-not-to-install scenarios more broadly, but the context needed to pick and use this tool is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_instancesARead-only
List installed local iPhone instances and their read-only status. This inventory is not a list of paired/connected devices. For recovery_owner=unconfigured, inspect xcrun devicectl list devices and run iphone-use doctor on the daemon host; ask which phone when ambiguous. Does not start, reconnect or claim a phone. Pass instance in any phone_* call to choose one; omitted instance preserves this MCP server's configured daemon. Missing/invalid instances never fall back to another phone.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the description's real contribution is the no-side-effect boundary ('does not start, reconnect or claim') and the non-fallback rule for missing/invalid instances, which prevents a dangerous silent substitution. It stops short of describing the inventory's shape or what read-only status values look like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and negative scope, and most sentences carry operational weight. It is dense and slightly instruction-heavy for a zero-argument listing tool, but nothing is filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-parameter, no-output-schema read tool, the description covers what is listed, what it is not, failure remediation, side-effect boundaries, and how its results feed into sibling calls. An agent has everything needed to call it and interpret the result correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4 and the schema needs no elaboration. The description's discussion of 'instance' concerns sibling phone_* calls rather than this tool's own inputs, which is useful context but not parameter documentation for this call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List installed local iPhone instances') plus the key attribute returned ('read-only status'), then immediately distinguishes this inventory from paired/connected devices so it can't be confused with device-discovery siblings like phone_status or phone_reconnect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly sets boundaries ('Does not start, reconnect or claim a phone'), gives a remediation path for recovery_owner=unconfigured (xcrun devicectl list devices, iphone-use doctor on the daemon host), and tells the agent to ask which phone when ambiguous. It also explains the downstream convention of passing instance to any phone_* call.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_jev_runADestructive
Hand ONE whole goal to a fast on-phone agent instead of driving it step by step: TypeSafe's Jev picks each next operation and target from the screen's element table (~1–2 s per step instead of a model turn), a small LLM types field values. Use it for a clear multi-step goal with no saved flow (check phone_flow_list first). It stops with status done / blocked / max_steps and the step history; it answers BLOCKED rather than send, pay, delete or share anything the goal did not explicitly ask for — still confirm such goals with the user first. Verify the end screen yourself. Needs TYPESAFE_API_KEY (or ~/.config/typesafe/key) and, for typing, TEXT_MODEL_API_KEY (or ~/.config/openrouter/key).
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | Bundle id to launch first, e.g. `com.apple.Preferences`. | |
| goal | Yes | The whole goal in plain language, with every value it needs ("In Settings, turn on Low Power Mode"). Jev never invents personal data. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| max_steps | No | Step budget (default 30). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by naming the terminal states (done / blocked / max_steps), returning step history, the ~1–2 s per-step cost profile, and the safety behavior of answering BLOCKED rather than sending, paying, deleting or sharing unrequested actions. That last point is the real value-add given destructiveHint=true and openWorldHint=true. Missing only finer detail such as what happens on a mid-run crash or how step history is shaped, which keeps it at 4 rather than 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core distinction and the alternative check, then cost, then outputs, then safety, then credentials — a sensible priority order with no filler sentences. It is dense with parentheticals (key paths, timing) and runs long for a single prose block, costing it a point on readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by declaring the terminal statuses and that step history comes back. For an autonomous, non-idempotent, destructive-capable agent tool it also covers credentials (TYPESAFE_API_KEY, TEXT_MODEL_API_KEY with file fallbacks) and verification expectations, leaving no obvious gap for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter already carries a substantive description (app bundle id, instance with no fallback, max_steps default 30), so the schema does the heavy lifting. The description reinforces the goal contract ('with every value it needs', never invents personal data) and clarifies max_steps via the 'stops with ... max_steps' phrasing, but adds no format or syntax detail beyond the schema — baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb and resource ('hand ONE whole goal to a fast on-phone agent') and immediately contrasts it with step-by-step driving, which is exactly the axis on which it differs from siblings like phone_run_steps and phone_tap_element. The mechanism (Jev picks the next operation from the screen's element table, a small LLM types values) makes the tool's role unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use ('a clear multi-step goal with no saved flow'), an explicit prerequisite check ('check phone_flow_list first'), and explicit user-facing caveats ('confirm such goals with the user first', 'Verify the end screen yourself'). It routes the agent away from this tool when a saved flow exists and tells it what to do after the call.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_keyADestructive
Press a navigation or editing key in the foreground app, acting on its current focus. Use phone_type for literal text and phone_tap_element to focus a field first. Check phone_status.drivable and the focused control: delete/backspace can erase content, and return/enter may submit or send it. The schema lists accepted names; unsupported names fail before input is sent. observe defaults on and returns the settled screen change. On an unknown outcome, inspect the screen before another key; repeating a key can change or submit content twice.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Supported names: `return`/`enter`, `escape`, `space`, `tab`, `delete`/`backspace`, `up`, `down`, `left`, `right`. | |
| observe | No | Default true: return what the screen settled to (as on phone_tap). `false` for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=false and openWorldHint=true, yet the description still adds real context: unsupported names fail before any input is sent, observe defaults on and returns the settled screen, and repeating a key can submit or erase content twice. It stops just short of describing per-key effects in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, no filler. The core action is front-loaded, followed by alternatives, then hazards, then the failure/observe behavior — each sentence carries distinct operational value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, non-idempotent input tool with no output schema, the description covers preconditions, side effects, return behavior ('the settled screen change'), and recovery on ambiguous results. Nothing an agent needs to invoke it safely is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description earns an extra point by explaining what 'observe' returns and by adding validation behavior for 'name' (unsupported names fail before input is sent) beyond the schema's accepted-name list.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('press a navigation or editing key') plus scope ('in the foreground app, acting on its current focus'). It actively distinguishes itself from siblings by naming phone_type and phone_tap_element and stating what each is for.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing: use phone_type for literal text, phone_tap_element to focus a field first, and check phone_status.drivable prior to acting. It also gives a when-not rule for retries ('on an unknown outcome, inspect the screen before another key').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_loginADestructive
Log the app on screen in with the user's OWN account from their password vault (bitwarden-use on the Mac). The daemon reads the vault entry and types it into the phone, so no password or username passes through this conversation. Use it when an app shows its login page during a task the user asked for. The entry is matched to the app; when several match, the answer lists their names: ask the user, then pass item (and user when names repeat). Existing-account login only: never sign-up, password changes or payment. When the answer has needs_code, the app sent a verification code: call again with code_via (sms or mail) and code_from (the sender) once it has arrived; at most two code requests per login. A push approval on another device needs the user. Read the screen afterwards to confirm the login.
| Name | Required | Description | Default |
|---|---|---|---|
| item | No | Vault entry name or id, when several entries match the app. | |
| user | No | Username, to pick between entries with the same name. | |
| submit | No | Tap the app's Log in button after filling (default true). | |
| code_via | No | Enter a verification code instead of logging in: sms, mail, totp or auto. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| code_from | No | Sender, brand or subject fragment of the message with the code. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial context beyond the annotations: no password or username traverses the conversation, the entry is matched to the app, ambiguity requires user input, at most two code requests per login, push approval must be handled by the user, and the screen should be read afterwards to confirm. That is real behavioral disclosure for a destructive, open-world mutation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action and credential-safety point, then layers the exclusions and code flow. Dense but every sentence carries operational content; slightly long for a single paragraph.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description compensates by describing the answer shapes the agent must react to (a list of matching entry names, needs_code) and the required follow-up of reading the screen. An agent has enough to call it and handle the common branches without guessing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds when-and-why for `item` and `user` (disambiguation after a name list is returned), when to pass code_via/code_from (only in the needs_code case), and the retry limit. It does not re-explain submit or instance, which the schema already covers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: logs an app on screen in using the user's own vault entry, with the daemon doing the typing. This is clearly distinct from siblings like phone_type or phone_tap, which it does not overlap with.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit trigger ('when an app shows its login page during a task the user asked for'), explicit exclusions ('never sign-up, password changes or payment'), and a fallback path when several entries match (ask the user, then pass item/user). The verification-code path and the two-request cap are also spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_logsA
Capture and read the paired phone's live system log for app debugging; use phone_crashes for saved crash reports instead. Start capture, reproduce the issue, then read (the default); stop releases the relay, while clear discards the in-memory buffer. Requires a reachable, paired device; connection/auth failures are errors, not an empty log. Capture self-stops after 15 minutes without reads. Filters apply to read results; pass the returned next cursor as since for incremental reads. Output lines are #seq time process[pid] level: message; error includes faults. Logs can contain private app data: collect only what the task needs.
| Name | Required | Description | Default |
|---|---|---|---|
| level | No | read: this level or more severe: debug, info, notice, error, fault. | |
| limit | No | read: at most this many lines (default 50). | |
| since | No | read: lines after this cursor — the previous read's `next`. | |
| action | No | `start` capturing the phone's system log, `read` (default) what was captured, `stop`, or `clear` the buffer. Capture stops by itself when nobody reads for a while. | |
| process | No | read: only this process (exact name, case-insensitive). | |
| contains | No | read: only messages containing this text (case-insensitive). | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses important operational behavior: a reachable paired device is required, connection or auth failures are errors rather than empty logs, capture self-stops after 15 minutes without reads, and clear discards the in-memory buffer. It also documents the output line format, including that error output contains faults, and warns that logs may contain private app data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loads the core purpose and the key sibling alternative. Subsequent clauses cover the action lifecycle, failure behavior, timeout, cursor workflow, output format, and privacy warning without obvious filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a multi-action tool with seven optional parameters and no output schema, the definition supplies the missing operational context: action lifecycle, default behavior, timeout, error semantics, cursor usage, output line format, and privacy considerations. Combined with the annotations, it is complete enough for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is already 100%, so the parameter-level documentation is strong without the description. The description still adds cross-parameter meaning by clarifying that filters apply to read results and that the previous read's next cursor should be supplied as since for incremental reads.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: capture and read the paired phone's live system log for app debugging. It also explicitly distinguishes this tool from the sibling phone_crashes for saved crash reports, so an agent can route correctly without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear workflow: start capture, reproduce the issue, then read as the default; stop releases the relay and clear discards the buffer. It names the alternative phone_crashes and the condition that should select it, and explains that filters apply only to read results and that the returned cursor should be passed as since for incremental reads.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_metricsARead-only
Read task performance aggregates for this MCP client's X-Phone-Owner on the selected instance. Includes recorded HTTP-call counts, batch/flow usage, stale/unknown outcomes, losses and p50/p95 call latency. HTTP calls are not model turns or token usage; model_round_trips remains null without a complete declared trace. Read-only: no phone connection, Runner launch or control lease. Default latest 5 open and recent runs, maximum 20 per section and a shared 12 KiB row budget; omitted counts and truncated state are explicit. Never reconnect solely for metrics. Global cross-owner loss counters are excluded.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Latest runs per section (open/recent), default 5, maximum 20. A shared 12 KiB row budget can reduce the number returned further. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint/openWorldHint annotations, it discloses that no phone connection, Runner launch, or control lease is acquired, that a shared 12 KiB row budget may silently reduce results with truncation explicitly flagged, and that some fields are intentionally null. These are exactly the operational traits an agent cannot infer from annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core purpose in the first sentence and groups caveats afterward; nearly every clause carries distinct information (limits, null semantics, no-reconnect rule, excluded counters). It is dense but slightly overloaded with hedging clauses in a single paragraph.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the necessary work of enumerating what the aggregate contains and how truncation appears in the response. It does not describe the shape of the returned rows (per-run vs. per-section fields), leaving a small gap for a reporting tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are already documented, including the limit default of 5, maximum 20, and the shared 12 KiB budget note. The description's mention of 'Default latest 5 open and recent runs, maximum 20' and 'selected instance' largely restates the schema rather than adding new syntax or constraints, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Read task performance aggregates') scoped to the caller's X-Phone-Owner and a selected instance, then enumerates the exact contents (HTTP-call counts, batch/flow usage, stale/unknown outcomes, losses, p50/p95 latency). An agent can tell this apart from phone_logs or phone_status without opening the schema, though no sibling is named directly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit exclusions and boundaries: 'Never reconnect solely for metrics' steers away from phone_reconnect, and 'Global cross-owner loss counters are excluded' plus 'model_round_trips remains null without a complete declared trace' clarify when the data will be absent. It gives clear context but does not name a positive alternative tool for adjacent needs (e.g., raw logs).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_read_valueARead-only
Read one current element's AX value without tapping, focusing, waking or taking an owner lease. Give exactly one identifier or exact label from a fresh screen, optionally kind; ambiguity refuses. Use this when phone_elements marks value_truncated or the task needs full field text. Returns one bounded value in text with source/completeness metadata; complete means the AX attribute was not cut by this API, not that the app exposes its full document. A smaller max_utf16 returns truncated:true. Secure fields refuse; old runners without ipu:valueRead refuse rather than promote a preview. Owner/privacy changes withhold the value.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | ||
| label | No | ||
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| max_utf16 | No | Maximum UTF-16 units returned (1–65536, default 8192); emoji may use two. | |
| identifier | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond readOnlyHint/openWorldHint it discloses real behavioral traits: no focus/wake/lease side effects, refusal on secure fields, refusal on old runners lacking ipu:valueRead instead of promoting a preview, and value withholding on owner/privacy changes. It also warns that 'complete' means only that the AX attribute was not cut by this API, not that the app exposed its full document — a genuinely useful caveat. The only gap is the exact shape of the source/completeness metadata it returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and its non-effects, then selection rules, then use-case routing, then return/refusal semantics. Dense and largely waste-free, though a few clauses are packed tightly enough to slow parsing on first read.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, selection preconditions, truncation semantics, refusal paths, and a return-value sketch even without an output schema. Minor gaps remain around accepted 'kind' values and the structure of the returned metadata, but nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 40% schema coverage the description carries extra weight: it defines the identifier-vs-label selection rule, requires freshness of the screen, notes kind is optional, and explains that a smaller max_utf16 yields truncated:true. It leaves 'kind' without any value guidance and never mentions the instance parameter, but the schema documents instance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Read one current element's AX value') plus a precise non-effect list (no tapping, focusing, waking, owner lease), which separates it from phone_elements and phone_tap_element in the sibling set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit trigger ('when phone_elements marks value_truncated or the task needs full field text'), the selection precondition (exactly one identifier or exact label from a fresh screen), and the refusal rule for ambiguity. The alternative is named directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_reconnectA
Start/restart on-device automation for the configured phone. This occupies the phone and may require the operator to unlock it. Use only to continue a current user-requested phone operation or screen/UI read that cannot proceed; never reconnect for initialization, status/health checks, a completed task, or to keep the phone ready. Idle release is intentional. Require phone_status released/offline and recovery_owner=daemon; inspect hint/setup_blocked_on first. Do not reconnect while releasing/reconnecting or a blocker remains. Once these conditions are met, call once, then wait for the outcome: pass wait_secs (e.g. 180) to block until drivable=true, a blocker (setup_blocked_on), or the timeout, instead of polling phone_status; wait.outcome is drivable, blocked, stopped or timeout. While it is reconnecting, report setup_phase/setup_message and obey setup_blocked_on. This tool never accepts a UDID and cannot switch devices. An externally managed runner returns an operator-owned recovery error.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| wait_secs | No | Wait up to this many seconds (1–300) for the outcome instead of returning at once: the call returns when the phone is drivable, a blocker appears (setup_blocked_on, e.g. the passcode), or the time is up — with the compact final state (a blocker other than the passcode must hold for 20 s first). A runner rebuild after an upgrade takes 1–2 min, so 180 covers it. Omit to return immediately. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only say readOnly=false, idempotent=false, openWorld=true, destructive=false; the description adds substantial context beyond that — the phone is occupied, an operator may need to unlock it, idle release is intentional, reconnecting has observable setup_phase/setup_message, and an externally managed runner yields an operator-owned recovery error. It stops just short of describing failure messages in full, but the coverage is well above the annotation bar.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose, then preconditions, then the call-once-and-wait protocol, so an agent can act on the first sentences. It is dense and slightly repetitive ('Idle release is intentional' plus the repeated blocker/releasing prohibitions), but nearly every clause encodes a distinct rule.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by naming wait.outcome states and the blocking conditions under which the call returns. For a state-mutating, operator-dependent tool, setup/recovery/error behavior is all covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description still adds value by recommending wait_secs=180 with a rationale (runner rebuild after upgrade takes 1–2 min), and by naming the wait.outcome values (drivable, blocked, stopped, timeout) that the schema does not enumerate. The instance parameter is left entirely to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — 'Start/restart on-device automation for the configured phone' — and immediately bounds scope with 'This tool never accepts a UDID and cannot switch devices,' which separates it from device-selection or multi-device siblings like phone_instances.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly enumerates when NOT to use it (initialization, status/health checks, completed task, keeping the phone ready) and names the alternative implicitly (phone_status for status). It also gives hard preconditions: phone_status must be released/offline, recovery_owner=daemon, inspect hint/setup_blocked_on first, and no reconnect while releasing or blocked.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_recordA
Record the iPhone screen to an MP4 on the daemon's Mac: start, perform the requested task, then stop to get path, duration and size. status checks the recording without starting one. Unlike phone_screenshot (one image) or phone_screen (live panel), this persists video under the instance state directory's runs/recordings. Requires a usable device video stream and a writable output directory. Auto-stops after 10 minutes or 200 MB; ended explains why. Video can contain private screens: record only the authorized task, stop when done, and do not treat the returned Mac path as an upload.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | `start` recording the screen to an MP4, `stop` and save it, or `status`. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial behavior beyond the annotations: auto-stop at 10 minutes or 200 MB, 'ended' explains why a run stopped, the daemon-side storage location under runs/recordings, the required device stream and writable output, and an explicit privacy/security caution about screen contents and the returned Mac path. Nothing here merely restates readOnlyHint/openWorldHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and lifecycle are front-loaded in the first clause, with differentiation and safety notes after. It is dense and packs several clauses into one paragraph, but almost every clause carries operational value; only the privacy admonition is marginally longer than needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still covers what the agent gets back (path, duration, size, 'ended' reason), the failure prerequisites, the size/time caps, and where files land. That is everything needed to invoke and interpret the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with action values ('start', 'stop', 'status') already documented in the schema and the instance fallback behavior described there. The description adds no additional syntax or format detail for either parameter, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Record the iPhone screen to an MP4') plus the lifecycle (start / task / stop), and explicitly contrasts itself with phone_screenshot (one image) and phone_screen (live panel). An agent can pick this over its siblings without reading any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives the operating sequence (start, perform task, stop to get path/duration/size), says when to use 'status' instead ('checks the recording without starting one'), and names the alternatives plus the condition that distinguishes them. Prerequisites (usable video stream, writable output dir) are also stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_release_ownerAIdempotent
Release this session's phone lease when its task ends, so another session can drive immediately. Unlike phone_hold(secs=0), this gives up ownership rather than merely clearing a pause. It sends no phone input and returns ok with owner=null; releasing an already-unowned phone succeeds. A different session's active lease returns phone_owned and is never taken over. Authentication/transport failures are errors; check phone_status.owner before assuming the lease was released.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing the success shape (ok with owner=null), that releasing an already-unowned phone succeeds, the exact failure mode for another session's lease (phone_owned, never taken over), and that auth/transport failures surface as errors. It even recommends a verification step via phone_status.owner. This is fully consistent with readOnlyHint=false and idempotentHint=true and adds real context on top.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four tight sentences, front-loaded with the primary action and its effect, then the disambiguation, then outcomes and error semantics. No filler or redundancy; every sentence carries distinct information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries return-value duty and does so (ok/owner=null vs phone_owned), plus error behavior and a verification hint. For a single optional-parameter mutation tool, nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single optional 'instance' parameter is fully documented in the schema, so the description need not repeat it. The description adds no parameter meaning of its own, which is the baseline 3 case.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (release) plus resource (this session's phone lease) and the triggering condition (when its task ends). It explicitly positions itself against the sibling phone_hold(secs=0), so an agent can distinguish the two without reading either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the alternative (phone_hold with secs=0) and explains exactly how they differ in effect: giving up ownership versus clearing a pause. It also implies when release is appropriate (task ends, wanting another session to drive immediately).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_run_endA
Close a live task opened with phone_run_start and return its metrics: HTTP calls (not model turns), batches, observed actions, flows, stale/unknown outcomes, failures and p50/p95 call time. Use the same run_id and session; a missing or already-closed run cannot produce a new live summary. Supply turn_ids only for a complete_trace declaration, including every model generation; otherwise model_round_trips remains unknown. This ends accounting only, not a video recording or the owner lease: use phone_record(stop) and phone_release_owner for those.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| turn_ids | No | Every model turn id of the task, when phone_run_start declared a complete trace; `[]` for a task that used no model turns. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already define the safety profile (readOnlyHint=false, destructiveHint=false, idempotentHint=false). The description adds important behavioral context beyond annotations: closing is accounting-only, does not stop video recording or release the owner lease, and an already-closed run produces no new summary. It could say more about failure modes or return shape, but it meaningfully exceeds the annotation baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and return metrics, then adds usage constraints and sibling alternatives. It is dense but every sentence carries useful information; minor wording complexity in the metrics list keeps it from being maximally crisp.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a non-idempotent close/accounting tool with no output schema, the description is complete enough: it names the metrics returned, explains the required run/session linkage, covers the special turn_ids condition, and clarifies what the tool does not do relative to phone_record and phone_release_owner.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, with run_id undocumented in the schema. The description compensates by explaining that the same run_id and session must be used, and it clarifies turn_ids semantics: supply them only for a complete_trace declaration, include every model generation, and otherwise model_round_trips remains unknown. The instance parameter is left to the schema, which is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: close a live task opened with phone_run_start and return its metrics. It distinguishes the tool from siblings by naming phone_run_start, phone_record(stop), and phone_release_owner.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit when-to-use and when-not-to-use guidance: use the same run_id and session, a missing or already-closed run cannot produce a summary, supply turn_ids only for a complete_trace declaration, and use phone_record(stop) and phone_release_owner for recording and lease termination.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_run_startA
Open a task boundary for this session's daemon metrics; pair with phone_run_end, not phone_record (video). Later session calls count toward this run; this does not drive or reserve the phone. Optional: without it, idle gaps infer runs. Use a fresh run_id for each task; starting the same still-open id resumes that run rather than resetting its counts. complete_trace is valid only before its first counted call and commits you to supply every model turn id at run_end. Returns a start acknowledgement; report errors rather than claiming metrics were recorded.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | A short id for this task: 1–64 of A-Z a-z 0-9 . _ : @ - | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| complete_trace | No | Set only if you will pass the id of EVERY model turn of this task to phone_run_end (including turns that made no tool call); only then are model round trips reported. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare this is not read-only, not idempotent, not destructive, and not open-world; the description adds substantial context beyond that. It clarifies that the tool does not drive or reserve the phone, how later calls count toward the run, what happens when reusing an open run_id, and what complete_trace commits the caller to.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with its core purpose, then packs only relevant operational details into a compact paragraph. Each clause adds distinct value: alternative routing, optionality, run_id reuse, complete_trace timing, and error-reporting behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description correctly states what is returned: a start acknowledgement and the need to report errors rather than assume metrics were recorded. It also covers the tool's operational role, optionality, and key constraints, making the definition complete for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameter documentation already carries much of the burden. The description still adds meaning beyond the schema by explaining run_id freshness versus resuming an existing open run, and by adding the timing constraint for complete_trace that the schema alone does not state.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Open a task boundary for this session's daemon metrics.' It immediately distinguishes this from sibling tools by naming phone_run_end and explicitly excluding phone_record (video), so an agent can identify the correct tool without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit pairing guidance ('pair with phone_run_end, not phone_record'), states that it is optional, and explains the fallback behavior ('without it, idle gaps infer runs'). It also covers run_id reuse semantics and the complete_trace timing constraint, leaving very little to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_run_stepsADestructive
THE DEFAULT WAY TO ACT: run a sequence of actions in ONE call and get the screen it ended on back in the same reply (observe, on by default) — one turn instead of one per tap. The daemon validates the whole sequence first, holds one control lock, and stops at the first failure. Use it as soon as a segment is understood; keep single-action tools for exploring an unknown screen. Step kinds: launch_app, open_url, tap_locator, tap_label, tap, type, key, shortcut, scroll, swipe, drag, longpress, multi_longpress (2–5 explicit normalized points held simultaneously), back, alert (system alerts; taps on them do not act), picker, perform (toggle a switch, adjust a slider: element+snapshot from phone_elements), wait_for, pause. Drag points can dwell at an edge while keeping one finger down. Guard every screen change with wait_for; never batch an irreversible step you have not verified. These step objects are exactly a saved flow's steps — when the response carries flow_suggestion, ask the user whether to keep the task as a flow (phone_flow_draft). The result names completed/applied counts and the failed step; retry_safe=false means DO NOT replay.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | Ordered steps. The daemon validates the whole list before sending the first action and stops immediately when any action or wait condition fails. | |
| observe | No | Return the screen the batch ended on (`snapshot`, `elements`, `alert`) in the same reply, so the next decision needs no phone_elements call. Default true; set false only when you will not look at the result. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only declare destructive/openWorld/non-idempotent; the description goes well beyond them by describing the execution model — whole-sequence pre-validation, a single control lock, stop-at-first-failure — and the risk semantics of retry_safe and unverified irreversible steps. It also discloses that alert-button taps are acknowledged without effect, which is a non-obvious trap an agent could not infer from the schema. Nothing here contradicts the destructiveHint=true annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The opening sentence is front-loaded with the payoff (one call, screen returned, one turn) and the constraints follow in descending priority. It is a single dense block, though; the long comma-separated enumeration of step kinds would scan faster as a list, and a few clauses (e.g. the multi_longpress finger-count restatement) duplicate schema text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool whose schema is a 19-variant oneOf, the description supplies exactly what the schema cannot: batching strategy, failure semantics, the safe-retry rule, and what the result reports (completed/applied counts and the failed step). With no output schema present, describing the result shape and the flow_suggestion follow-up closes the remaining gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, so the baseline is 3, but the description adds real meaning: it enumerates the step kinds so an agent can plan a batch from the description alone, clarifies multi_longpress as 2–5 simultaneous fingers, explains drag edge-dwell, and states that these step objects are byte-identical to a saved flow's `steps`. It also names the one non-obvious workflow dependency, that `perform` needs element+snapshot from phone_elements.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — run a sequence of actions in one call — and immediately names the differentiator from siblings: returns the ending screen in the same reply, one turn instead of one per tap. The lead 'THE DEFAULT WAY TO ACT' plus the contrast with single-action tools lets an agent place phone_run_steps relative to phone_tap, phone_type, and phone_elements without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('as soon as a segment is understood') and when-not ('keep single-action tools for exploring an unknown screen'), plus hard operational rules: guard every screen change with wait_for, never batch an unverified irreversible step, and treat retry_safe=false as DO NOT replay. It also handles the follow-up branch by telling the agent to ask the user about a flow when flow_suggestion appears.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_screenARead-only
Open the live iPhone screen panel beside the chat, or reuse the one already open, in hosts that render MCP Apps: the person watches the phone while you work and can press Home or refresh. Read-only: takes no owner lease and wakes nothing. Returns a short status (phone, drivable, owner). Open it once when the person wants to watch; it never replaces phone_elements or phone_screenshot for your own observation. Hosts without MCP Apps show no panel.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint/openWorldHint annotations: states it takes no owner lease, wakes nothing, returns a short status with named fields (phone, drivable, owner), and degrades to no panel on non-MCP-Apps hosts. These are non-obvious side-effect and failure-mode facts the annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded in the first clause and every sentence carries distinct information (reuse, side effects, return, exclusion, host limit). The opening sentence is somewhat clause-stacked, which costs a bit of readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description names the returned status fields, and it covers the host-compatibility caveat and the read-only/no-lease behavior. Nothing an agent needs to decide or call correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'instance' parameter is fully documented in the schema, so a baseline 3 applies. The description adds nothing about instance selection or its "no fallback on failure" behavior beyond what the schema already says.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ("Open the live iPhone screen panel beside the chat") with the reuse semantics spelled out, and it explicitly distinguishes itself from phone_elements and phone_screenshot. An agent can tell immediately this is a UI-panel tool, not an observation tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit positive trigger ("Open it once when the person wants to watch"), an explicit exclusion ("it never replaces phone_elements or phone_screenshot for your own observation"), and a host precondition ("in hosts that render MCP Apps").
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_screen_frameARead-only
Internal frame transport for the iphone-use MCP Apps panel; models use phone_screenshot for a still image and phone_elements for text. Returns structuredContent with status, frame (mime_type=image/png and base64 data), ok and error. A private hold returns frame=null; an empty or failed capture is reported rather than invented pixels. max_side is clamped to 200–1000; fresh bypasses the reusable frame. The panel calls this with visibility app; do not poll it from model turns or use it for health checks.
| Name | Required | Description | Default |
|---|---|---|---|
| fresh | No | Capture now instead of reusing the settled or live frame. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| max_side | No | Longest side of the frame in pixels (200–1000, default 600). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond readOnlyHint/openWorldHint, it discloses the return payload (status, frame with mime_type and base64 data, ok, error), the private-hold behavior returning frame=null, the refusal to fabricate pixels on failed capture, and that fresh bypasses the reusable frame. That is rich behavioral context an agent cannot infer from annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four dense sentences, front-loaded with the purpose and the sibling routing before return details and usage restrictions. Information-dense with minimal waste, though the return-field enumeration is slightly list-like.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description fully covers the return shape, error/empty-capture semantics, and the calling convention, leaving nothing an agent needs missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description largely restates what the schema already says (max_side clamped to 200–1000, fresh bypasses reuse). It adds little new per-parameter meaning, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Internal frame transport') and immediately distinguishes itself from siblings by naming phone_screenshot for stills and phone_elements for text. An agent can identify the tool's role and its alternatives without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when/when-not guidance: the panel calls it with visibility app, models should not poll it from model turns, and it must not be used for health checks. It also routes to the correct siblings for image and text needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_screenshotARead-only
Read one iPhone screen as an image/png content block when pixels matter; use phone_elements for text and semantic targets, or phone_screen to let the person watch a live MCP Apps panel. It reuses the captured settled frame after an observed action. max_side defaults to 1200; 0 requests full resolution. overlay=true draws interactive frames and their phone_elements indexes. Capture only for the current user task; private holds or offline devices can refuse it. Follow phone_status guidance instead of reconnecting for health checks. An image alone does not prove an action succeeded; check the target state or receipt.
| Name | Required | Description | Default |
|---|---|---|---|
| overlay | No | Draw each interactive element's frame and its phone_elements index on the image, to match what you see with the rows you can tap. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| max_side | No | Longest side of the image in pixels. Default 1200, which reads fine and costs ~0.9k image tokens instead of ~1.5k at full resolution; ask for 0 (full resolution) only to read very small text. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond readOnlyHint/openWorldHint the description discloses non-obvious behavior: the frame is the settled capture reused after an observed action, overlay draws element frames with their phone_elements indexes, private holds or offline devices can refuse the call, and the ~0.9k vs ~1.5k image-token cost. The verification caveat is exactly the kind of behavioral warning annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose and the sibling routing before any caveats, with no filler sentences. It runs long with several semicolon-joined constraints, but each carries distinct information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only capture tool with no output schema, the description still states the return shape (image/png content block), the token economics, the failure modes, and the limits of what the result proves. Nothing essential for calling it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains overlay, instance, and max_side thoroughly. The description's restatement of the 1200 default and 0-for-full-resolution largely duplicates that, though tying overlay to phone_elements indexes adds a little cross-tool meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Read one iPhone screen as an image/png content block when pixels matter" gives a specific verb, resource, return type, and the exact selection condition. It immediately distinguishes itself from phone_elements (text/semantic) and phone_screen (live panel), so an agent can route without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit alternatives and their selection criteria are named: phone_elements for text/semantic targets, phone_screen for the human-visible live panel, phone_status for health checks instead of reconnecting. It also adds a scope constraint ("capture only for the current user task") and the caveat that an image alone does not prove success, with a redirect to check target state or receipt.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_scrollADestructive
Move the current viewport with one device-side swipe; use phone_scroll_find to locate a known label, or phone_collect_list to read multiple pages. x,y are a normalized anchor in the intended scrollable region. Avoid text fields, sliders and screen edges: a misplaced gesture can edit a control or navigate away. Positive dy reveals content below, negative above; dx reveals right/left. Travel is 1.5 times each delta in points, clamped to 15–75% of that screen dimension. Requires phone_status.drivable=true. observe defaults on and returns the settled screen change; inspect it before repeating a swipe.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | Horizontal anchor position, normalized 0–1. | |
| y | Yes | Vertical anchor position, normalized 0–1. | |
| dx | Yes | Horizontal scroll delta. Positive reveals content to the right. | |
| dy | Yes | Vertical scroll delta. **Positive dy reveals content farther down**; negative dy reveals content above. ~80 ≈ 15% of a screen, ~400 ≈ 75%. | |
| observe | No | Default true: return what the screen settled to (as on phone_tap). `false` for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as non-read-only, open-world, non-idempotent, and destructive; the description adds concrete behavioral context: the prerequisite phone_status.drivable=true, the risk that a misplaced gesture can edit a control or navigate away, the travel scaling/clamping behavior, and the default observe return behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loaded: it begins with the core action and alternatives, then groups usage hazards, direction semantics, scaling, prerequisites, and observe behavior. Every sentence adds operational value for invoking a risky gesture correctly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive open-world gesture tool with no output schema, the description supplies what the agent needs: alternatives, prerequisites, risky regions to avoid, direction conventions, travel scaling, and observe behavior. The annotations provide the safety profile, and the description fills in the operational details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema by clarifying that x,y are an anchor within the intended scrollable region and by explaining that travel is 1.5 times each delta in points, clamped to 15–75% of that screen dimension. Direction semantics are also reinforced, though they are already largely present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action and resource: moving the viewport via one device-side swipe. It also names sibling alternatives (phone_scroll_find, phone_collect_list), so the agent can distinguish this tool from adjacent scrolling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes known-label lookup to phone_scroll_find and multi-page reading to phone_collect_list. It also warns against swiping text fields, sliders, and screen edges, and advises inspecting the settled screen before repeating a swipe.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_scroll_findA
Find one exact label, swiping at most max_swipes times (default 1, max 5) when it is not on screen. Stops AT ONCE — no further swipe — when the label is ambiguous (several on-screen matches: candidates with index and frame), covered by another control, or not drawn. Found: target (index, kind, frame) plus a snapshot for phone_tap_element. Not found: on-screen labels containing the text as candidates. No screenshot is taken; absence after the budget is not proof — look at the screen before swiping further. Optional snapshot (from a read of the target app; never filled in for you): a known foreground app mismatch refuses before any swipe (app_changed). Setup may already have affected the device. Stops when ownership or privacy authorization changes and withholds observations. swipes_confirmed counts Native ACKs, not displacement; swipes_uncertain means no ACK and must not be replayed automatically. Requires phone_status drivable=true.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | Narrow to one element kind (e.g. `Button`) when a text shares the label. | |
| label | Yes | Exact label to find. | |
| region | No | Swipe inside this normalized `[x, y, w, h]` area (default: whole screen). | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| snapshot | No | A snapshot from an earlier read of the target app: when given, a different foreground app refuses before any swipe. Not filled in for you. | |
| direction | No | `down` (default) or `up`: which way to look when the label is absent. | |
| max_swipes | No | Swipes allowed when the label is not on screen, 0–5 (default 1). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses real behavior beyond the annotations: immediate stop on ambiguity, covered, or not-drawn elements; no screenshot taken; swipes_confirmed counts Native ACKs rather than displacement; swipes_uncertain must not be replayed; observations withheld if ownership/privacy authorization changes. These additions are substantive, but several statements are compressed to the point of being cryptic ('Setup may already have affected the device'), so the disclosure is rich yet unevenly legible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The output/mode split ('Found:' / 'Not found:') is well front-loaded, but the block is a long run of terse fragments, some of which are elliptical enough to require re-reading ('Stops when ownership or privacy authorization changes and withholds observations'). A few sentences, notably the setup-affect note, lack a clear antecedent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description properly carries the return contract: `target` (index, kind, frame), `candidates` with index and frame, `snapshot` for phone_tap_element, plus swipes_confirmed/swipes_uncertain semantics and the app_changed refusal. It is nearly complete for a 7-parameter, open-world tool; only the ownership/privacy stop condition remains under-explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, and the description does add meaning for `snapshot` (never filled in for you; a foreground app mismatch refuses before any swipe with app_changed) and for `max_swipes` (default 1). Otherwise it largely restates what the schema already documents for label, kind, region, direction, and instance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening clause gives a specific verb and resource ('Find one exact label') plus the operating mode (swipe at most max_swipes times when not on screen), so an agent knows exactly what the call does. It also distinguishes itself from related siblings by naming phone_tap_element as the consumer of the returned snapshot and phone_status as a prerequisite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is strong: it warns 'absence after the budget is not proof — look at the screen before swiping further,' states the phone_status drivable=true prerequisite, and explains when a provided snapshot causes a pre-swipe refusal. It stops short of explicitly naming alternatives such as phone_scroll or phone_elements for non-label-driven scrolling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_set_locationAIdempotent
Simulate GPS for testing location-aware apps. Supply both lat (-90–90) and lon (-180–180), in finite decimal degrees, or clear=true without coordinates to restore real location; combining clear with coordinates is refused. Every app sees the simulated position until cleared; it is not a navigation or address lookup tool. Counts as one daemon action and requires authorized device control. Returns the applied location/clear status; on a transport failure the effect is unknown. Inspect the result before proceeding and clear the simulation when the test ends.
| Name | Required | Description | Default |
|---|---|---|---|
| lat | No | Latitude in decimal degrees (WGS84), -90..90. | |
| lon | No | Longitude in decimal degrees (WGS84), -180..180. | |
| clear | No | `true` (without lat/lon) returns the phone to its real location. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations, the description discloses that the simulated location affects every app until cleared, that the call counts as one daemon action, that authorized device control is required, and that a transport failure leaves the effect unknown. It also advises inspecting the result and cleaning up the simulation, which is exactly the behavioral context an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the purpose, then gives invocation rules, side effects, failure behavior, and cleanup guidance. Every sentence contributes useful information, and there is no redundant or filler text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description states that it returns the applied location or clear status and notes the uncertainty on transport failure. Together with the annotations and fully described schema, this is complete enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents the parameters, but the description adds meaningful semantics: lat and lon must be supplied together, clear=true must be used without coordinates, and combining clear with coordinates is refused. It does not add detail about the instance parameter, but the schema covers that adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: simulate GPS for testing location-aware apps. It also explicitly says the tool is not a navigation or address lookup tool, which helps an agent distinguish it from conceptually adjacent requests. The scope is clear without needing to inspect the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the primary use case, the two valid invocation modes (lat/lon together or clear=true alone), the refusal of mixed clear and coordinates, and the need to clear the simulation when testing ends. These are explicit when, when-not, and cleanup instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_shortcutADestructive
Use Home (home), Spotlight (spotlight), physical volume buttons (volume_up, volume_down), or request a system screenshot in Photos (save_screenshot). Volume acknowledgement confirms only a button request, not the volume level; Simulator and older Runners without ipu:volumeButtons refuse before dispatch. save_screenshot is a mutation, unlike phone_screenshot which only returns image bytes. Its receipt means requested, not saved_verified, and requires ipu:systemCaptureRequest. If iOS opens a screenshot preview, inspect the system UI and choose Done/Save to Photos for that existing image; never take another screenshot to complete its save. Check Photos or its picker before claiming it saved. Never replay an uncertain capture or volume request. Navigation leaves the current app or changes its visible state. Use phone_key for field keys, or phone_run_steps with launch_app for a specific app. Requires phone_status.drivable=true. Names are trimmed and case-insensitive. switcher and other names fail before input is sent; the runner cannot synthesize the App Switcher gesture. observe defaults on and returns the settled screen change. Verify it before another action; repeating system navigation may lose the current context.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Home, Spotlight, Photos capture (`save_screenshot`), or physical-only `volume_up` / `volume_down`. App Switcher is unsupported by the device runner. | |
| observe | No | Default true: return what the screen settled to (as on phone_tap). `false` for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructive/openWorld/non-idempotent, and the description adds the details that matter: receipts mean 'requested, not saved_verified', volume acknowledgement is not a level confirmation, navigation changes app state, and uncertain captures must never be replayed. This is exactly the extra context annotations cannot express, and it is consistent with destructiveHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a dense, run-on block rather than front-loaded clauses, and several points are stated twice – the 'requested vs saved_verified' receipt and 'check Photos before claiming saved' cover the same ground, as do 'navigation leaves the current app' and 'repeating system navigation may lose context'. Nearly every sentence earns its place, but the duplication costs it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, gated, open-world system-action tool with no output schema, the description covers valid inputs, refusal conditions, permission requirements, verification steps, and non-retry policy. Nothing an agent needs before invoking it is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds real meaning about the name values: they are trimmed and case-insensitive, 'switcher' and other names fail before input is sent, and each named shortcut has documented side effects. The observe default is explained consistently in both places, which is mild redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description enumerates the concrete shortcuts (Home, Spotlight, volume buttons, Photos screenshot) so the agent knows what the tool does despite the vague name 'shortcut'. It also distinguishes itself from phone_screenshot, phone_key, and phone_run_steps. The only weakness is that no single sentence states the purpose up front – the enumeration does the work.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing: 'Use phone_key for field keys, or phone_run_steps with launch_app for a specific app', plus 'save_screenshot is a mutation, unlike phone_screenshot which only returns image bytes'. It also states preconditions (phone_status.drivable=true, ipu:volumeButtons, ipu:systemCaptureRequest) and what fails before dispatch.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_statusARead-only
Read the phone's status without taking control. owner names the session currently driving the phone (this session presents IPHONE_USE_OWNER, else mcp-); if it is someone else, do not drive the phone — control calls will be refused with phone_owned. The JSON preserves backend, target_configured, managed_runner, managed_runner_pending, recovery_owner, device_state, screen_state, runner, runner_actionable, locked, drivable, released, hint, next_step, recovery, setup_blocked_on, setup_phase, and setup_message. Gate actions on drivable=true. If target_configured=false and recovery_owner=unconfigured, phone_instances is only installed instances: inspect xcrun devicectl list devices and run iphone-use doctor on the daemon host; ask which phone if ambiguous, then IPU_RUNNER_UDID= iphone-use setup for the confirmed target only when the task authorizes it. Preserve --instance for named instances. lock_readiness says whether the phone locks on its own when idle (passcode_protected, auto_lock_secs, keep_awake, verdict, hint): will_lock_needs_person means a person must unlock it after a pause; tell the owner the hint instead of retrying. For trust, distinguish Trust This Computer pairing from developer-certificate approval under iPhone Settings > General > VPN & Device Management; use setup_message evidence. A separate passcode/automation prompt may appear later. needs_passcode_on_phone records a possible authorization requirement, not a confirmed visible prompt. Relay next_step; never request a passcode, pause screenshots/elements during sensitive input, and check status after the person finishes. Do not loop reconnect. keep_runner_alive (non-null) means idle release keeps the runner up because its next start would need a person. For status/health checks, report the state and stop. Idle release is intentional: reconnect (phone_reconnect, once) only when a current user-requested task needs the phone, recovery_owner=daemon, and no blocker or reconnect is in progress; then poll until drivable=true.
| Name | Required | Description | Default |
|---|---|---|---|
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint already covering safety, the description goes far beyond: it discloses the owner/refusal semantics (phone_owned), the full field list preserved in the JSON, the keep_runner_alive meaning, that idle release is intentional, and passcode/trust subtleties. This is substantive behavioral context annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded, but the body is a dense wall of imperative sentences mixing routing, recovery, trust, and passcode guidance. Much is relevant, yet it's hard to scan and some prose could be compressed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only status tool with no output schema, the description enumerates the returned fields and explains gating, recovery, setup, and reconnect decisions. An agent has enough to interpret results and act correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds real nuance beyond the schema: 'Preserve --instance for named instances' and the no-fallback caveat, giving the agent usage guidance the schema doesn't.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource: 'Read the phone's status without taking control.' The 'without taking control' clause explicitly distinguishes it from the many driving/control siblings (phone_tap, phone_type, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use ('For status/health checks, report the state and stop') and when-not (reconnect only under a named set of conditions, do not loop reconnect). It also names the alternative sibling phone_reconnect and the exact preconditions for calling it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_tapADestructive
Tap a pixel target using normalized x,y from 0–1, with (0,0) at top-left. Prefer phone_tap_element for an accessible control; use phone_screenshot to locate a pixel-only target. Requires phone_status.drivable=true and current screen evidence: a tap may send, buy, delete or toggle whatever occupies that point, so authorize its actual effect first. observe defaults on and returns the settled screen change. Read outcome/retry_safe; if delivery is unknown, inspect the current state or receipt before considering another tap.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | Horizontal position, normalized 0–1 (0 = left edge, 1 = right edge). | |
| y | Yes | Vertical position, normalized 0–1 (0 = top edge, 1 = bottom edge). | |
| observe | No | On by default: the result carries what the screen settled to — what changed since your last look (`delta`), or the whole tree the first time — so you rarely need `phone_elements` afterwards. The settled screen is also captured but NOT sent: `phone_screenshot` returns it at once if you need to see it. `false` skips both for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial behavioral context beyond annotations: destructive effects of tapping arbitrary coordinates, prerequisite status, default observe behavior, settled screen change, and the need to read outcome/retry_safe before repeating an uncertain tap. This aligns with and enriches the destructiveHint, non-idempotent, open-world annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action and coordinate convention, then moves to sibling preference, prerequisites, safety warning, observe behavior, and retry guidance. Dense but every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, open-world tap tool with no output schema, the description includes prerequisites, safety expectations, observe behavior, outcome/retry_safe semantics, and guidance on inspecting state before retrying. Nothing essential for correct invocation appears missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents x, y, observe and instance thoroughly. The description reinforces the normalized coordinate system and observe default but does not add much parameter-level meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States the specific action (tap a pixel target), the coordinate system (normalized x,y from 0–1 with top-left origin), and names phone_tap_element as the preferred sibling for accessible controls. It clearly distinguishes pixel-target tapping from element-based tapping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to prefer phone_tap_element and when to use phone_screenshot to find a pixel-only target. It also states the prerequisite phone_status.drivable=true and warns that the agent must authorize the actual effect first because a tap can send, buy, delete or toggle.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_tap_elementADestructive
Tap one element by its zero-based index and snapshot token from the SAME phone_elements response. The daemon re-reads the tree and refuses the action if the UI changed, so a stale element reference cannot silently tap a different control. Use identifier/kind/label/state fields to choose the index. Snapshot refs are ephemeral: never persist them in a flow.
| Name | Required | Description | Default |
|---|---|---|---|
| element | Yes | Zero-based element index from `phone_elements`. | |
| observe | No | Default true: return what the screen settled to (as on phone_tap). `false` for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| snapshot | Yes | Snapshot token from the same `phone_elements` response. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
It explains that the daemon re-reads the tree and refuses the action if the UI changed, which adds safety context beyond the annotations. It also notes that refs are ephemeral. It does not explain why the tool is annotated destructiveHint=true, or what the refusal error looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each earning its place. The purpose is front-loaded, followed by the safety behavior, selection guidance and a persistence warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
It covers the prerequisite workflow, the staleness protection and the ephemeral refs, which is solid for a UI action tool. The return value is described only in the observe parameter. Sibling differentiation among the tap tools is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. The description adds that the index should be chosen from the identifier/kind/label/state fields, and that element and snapshot must come from the same response. This is meaningful guidance beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
It names a specific action (tap one element by index plus snapshot token) and ties it to phone_elements. It does not say how this differs from the siblings phone_tap (coordinates?) or phone_tap_label, so an agent choosing among the three tap tools must infer the distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear prerequisite: the index and token must come from the SAME phone_elements response. It also says how to pick the index and warns never to persist snapshot refs in a flow. It does not say when to prefer phone_tap_label or phone_tap over this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_tap_labelADestructive
Tap an iPhone UI element by an EXACT visible label (requires drivable=true in phone_status). Reads a fresh phone_elements snapshot, requires exactly one match, then performs a snapshot-bound tap. Zero or multiple matches return an error and send NO action. For duplicate labels, choose by identifier/kind/state from phone_elements and call phone_tap_element with that response's snapshot.
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | The element's visible accessibility label, exactly as shown by `phone_elements` (e.g. "新备忘录", "Connect"). | |
| observe | No | Default true: return what the screen settled to (as on phone_tap). `false` for the fastest bare action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare that this is a destructive, non-idempotent open-world write. The description adds materially: zero or multiple matches send NO action, the tap is bound to a specific phone_elements snapshot (staleness risk), and a fresh snapshot is read before acting. That is exactly the safety context an agent needs before a destructive tap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the action, then the failure contract, then the fallback path. Every clause carries either a constraint or a routing instruction; nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, non-idempotent tap with no output schema, the description covers the precondition (drivable status), the failure behavior, the snapshot-binding semantics, and the observe=true return notion. An agent has everything needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description still adds the crucial constraint that 'label' must resolve to exactly one match, otherwise the call errors. The instance/observe parameters are left entirely to the schema, which is acceptable given full coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Tap an iPhone UI element by an EXACT visible label') and implicitly separates itself from phone_tap (coordinate-based) and phone_tap_element (identifier-based). An agent can pick the right sibling without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit prerequisite ('requires drivable=true in phone_status'), the exact selection rule (one match only), and a named alternative for the ambiguous case: use phone_tap_element with the snapshot from phone_elements when labels duplicate. That is when-to-use, when-not, and where-to-go instead, all in one place.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
phone_typeADestructive
Insert literal Unicode text into the currently focused iPhone field. Use phone_tap_element to focus the intended field first; use phone_key for a navigation or editing key. Requires phone_status.drivable=true. Text can replace a selection or reach the wrong field if focus changed. Long text is chunked. Optional budget_ms (6000..15000) can stop at a safe boundary; continue only with the returned continue_token, without text/clear/budget_ms. Tokens are single-use and scoped to the original field, owner, privacy, lifecycle and session. Unknown outcomes never issue a token. On partial failure inspect characters_confirmed and remaining_text; never resend the whole value blindly. observe defaults on and returns the settled screen change. Verify the resulting field before submitting; an unknown outcome is not evidence that nothing was typed.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | Unicode text to start typing, up to 20000 scalars. Omit when continuing. | |
| clear | No | Clear the bound field first. Never combine with continue_token. | |
| observe | No | Default true: return the settled screen change after a completed action. | |
| instance | No | Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure. | |
| budget_ms | No | Optional typing budget, 6000..15000 ms, requiring a named owner and inputContinuation. | |
| continue_token | No | Single-use token from a safe chunk boundary. Pass without text, clear or budget_ms. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare the mutation/destructive profile; the description adds substantial depth beyond them: focus can shift and text can reach the wrong field, long text is chunked at safe boundaries, tokens are single-use and scoped to field/owner/privacy/lifecycle/session, and unknown outcomes never issue a token. It also discloses partial-failure recovery via characters_confirmed and remaining_text and warns that an unknown outcome is not evidence nothing was typed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and focus prerequisite are front-loaded, then budget/token mechanics, then failure handling and verification, in a logical order with no filler. It is dense and telegraphic to the point that a couple of clauses (token scoping dimensions) read as enumerated keywords, but every sentence conveys operational information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, non-idempotent, open-world typing tool with no output schema, the description covers what an agent needs before and after invocation: precondition gating, chunking and continuation, partial-failure inspection fields, and a verification instruction before submitting. Nothing material to calling it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds relational meaning the per-parameter descriptions do not carry on their own: the budget_ms/continue_token protocol, the incompatibility of tokens with text/clear/budget_ms, and the meaning of an absent token on unknown outcome. It does not spell out the `instance` or `clear` selection semantics, keeping it from a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Insert literal Unicode text into the currently focused iPhone field') and immediately distinguishes itself from phone_tap_element (focusing) and phone_key (navigation/editing keys). An agent can tell exactly what this tool does versus its siblings without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the precondition sequence (focus with phone_tap_element first), the alternative for keys (phone_key), and the gating requirement (phone_status.drivable=true). It also states the continuation path for budget_ms and the prohibition on reusing text/clear/budget_ms with a token, which is genuine when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
16 tool updates
v0.27.34- Changed
phone_apps2 fields changed- changed
Input schema / properties / country / descriptionPrevious value: -"App Store storefront for the store search, two letters. Default `cn`."New value: +"App Store storefront for the store search, two letters. Default: the\nstore the phone's apps came from (or IPHONE_USE_STORE_COUNTRY, else the\nRegion setting); `cn` only when none is known, with a warning. The\nanswer's `country_source` says which." - added
Input schema / properties / limit / minimumAdded value: +0
- Changed
phone_collect_list1 field changed- added
Input schema / properties / max_pages / minimumAdded value: +0
- Changed
phone_crashes1 field changed- added
Input schema / properties / limit / minimumAdded value: +0
- Changed
phone_elements8 fields changed- added
Input schema / $defs / ElementSelector / descriptionAdded value: +"A literal filter over exposed snapshot metadata.\n\n```\nlet selector: core::element_selector::ElementSelector =\n serde_json::from_str(r#\"{\"value_present\": true}\"#).unwrap();\nlet row = serde_json::from_str(r#\"{\"kind\":\"TextField\",\"value\":\"\"}\"#).unwrap();\nassert!(selector.validate().is_ok());\nassert!(selector.matches(&row));\n```" - added
Input schema / $defs / ElementSelector / properties / indexAdded value: +{ + "$ref": "#/$defs/MatchOrdinal", + "description": "Zero-based ordinal among matching AX rows, 0..199; not a snapshot row index.\nRequires other criteria and a proven tree-order rank." +} - added
Input schema / $defs / ElementSelector / properties / nameAdded value: +{ + "description": "Exact name: non-empty identifier when present, otherwise label; case-sensitive.", + "type": [ + "string", + "null" + ] +} - changed
Input schema / $defs / ElementSelector / properties / value / descriptionPrevious value: -"Exact exposed accessibility value; known truncated previews never match.\nAn empty string matches a present empty value."New value: +"Exact exposed accessibility value; truncated, unavailable and secure values are unknown, never matches.\nAn empty string matches a present empty value." - added
Input schema / $defs / ElementSelector / properties / value_presentAdded value: +{ + "description": "Whether the snapshot supplies a string value, including empty or truncated\nstrings. False matches an omitted/null non-secure snapshot value, not an App's\nnull AX value. Protected SecureTextField rows match neither boolean.", + "type": [ + "boolean", + "null" + ] +} - added
Input schema / $defs / MatchOrdinalAdded value: +{ + "description": "Zero-based ordinal among matches, never a returned snapshot row index.", + "maximum": 199, + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / limit / minimumAdded value: +0 - changed
Input schema / properties / selector / descriptionPrevious value: -"Literal case-sensitive field filters combined with AND; preserves original snapshot indexes.\nRequires at least one field; filters AX elements, not separate OCR text."New value: +"Literal case-sensitive field filters combined with AND; preserves original snapshot indexes.\nRequires literal criteria; optional index0..199 chooses one match in scoped/query tree order before limit.\nRow index remains the full snapshot index; incomplete or unknown earlier matches refuse an unproven rank.\nFilters AX elements, not separate OCR text."
- Changed
phone_hold1 field changed- added
Input schema / properties / secs / minimumAdded value: +0
- Changed
phone_home_layout1 field changed- added
Input schema / properties / page / minimumAdded value: +0
- Changed
phone_jev_run1 field changed- added
Input schema / properties / max_steps / minimumAdded value: +0
- Changed
phone_logs2 fields changed- added
Input schema / properties / limit / minimumAdded value: +0 - added
Input schema / properties / since / minimumAdded value: +0
- Changed
phone_read_value1 field changed- added
Input schema / properties / max_utf16 / minimumAdded value: +0
- Changed
phone_reconnect1 field changed- added
Input schema / properties / wait_secs / minimumAdded value: +0
- Changed
phone_run_steps7 fields changed- added
Input schema / $defs / PhoneDragPoint / properties / dwell_ms / minimumAdded value: +0 - added
Input schema / $defs / PhoneElementLocator / properties / label_containsAdded value: +{ + "description": "Case-sensitive literal substring of the normalized label.", + "maxLength": 1000, + "minLength": 1, + "type": [ + "string", + "null" + ] +} - changed
Input schema / $defs / PhoneStep / oneOfPrevious value: -[ - { - "additionalProperties": false, - "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Long-press normalized screen coordinates.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 600, - "type": "integer" - }, - "kind": { - "const": "longpress", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold 2–5 distinct normalized points simultaneously; array length is finger count.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 1000, - "type": "integer" - }, - "kind": { - "const": "multi_longpress", - "type": "string" - }, - "points": { - "items": { - "$ref": "#/$defs/PhoneHoldPoint" - }, - "type": "array" - } - }, - "required": [ - "kind", - "points" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Swipe once from one normalized screen point to another.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "kind": { - "const": "swipe", - "type": "string" - }, - "x1": { - "type": "number" - }, - "x2": { - "type": "number" - }, - "y1": { - "type": "number" - }, - "y2": { - "type": "number" - } - }, - "required": [ - "kind", - "x1", - "y1", - "x2", - "y2" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "hold_ms": { - "default": 500, - "type": "integer" - }, - "kind": { - "const": "drag", - "type": "string" - }, - "points": { - "default": null, - "items": { - "$ref": "#/$defs/PhoneDragPoint" - }, - "type": [ - "array", - "null" - ] - }, - "x1": { - "type": [ - "number", - "null" - ] - }, - "x2": { - "type": [ - "number", - "null" - ] - }, - "y1": { - "type": [ - "number", - "null" - ] - }, - "y2": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap one exact, unique accessibility label.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap_label", - "type": "string" - }, - "label": { - "type": "string" - } - }, - "required": [ - "kind", - "label" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "if_present": { - "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", - "type": "boolean" - }, - "kind": { - "const": "tap_locator", - "type": "string" - }, - "locator": { - "$ref": "#/$defs/PhoneElementLocator" - }, - "via": { - "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "locator" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "clear": { - "default": false, - "type": "boolean" - }, - "kind": { - "const": "type", - "type": "string" - }, - "text": { - "type": "string" - } - }, - "required": [ - "kind", - "text" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Send one supported device-native key.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "key", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Trigger Home or Spotlight.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "shortcut", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "dx": { - "type": "number" - }, - "dy": { - "type": "number" - }, - "kind": { - "const": "scroll", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y", - "dx", - "dy" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; a\nloose name that starts a word of exactly one installed app's name also\nlaunches it, and the step's `resolved` says which; an ambiguous or\nunknown name is refused with candidates, nothing sent).", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "app": { - "type": [ - "string", - "null" - ] - }, - "bundle": { - "type": [ - "string", - "null" - ] - }, - "fresh": { - "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", - "type": "boolean" - }, - "kind": { - "const": "launch_app", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "open_url", - "type": "string" - }, - "url": { - "type": "string" - } - }, - "required": [ - "kind", - "url" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Navigate back inside the current application.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "back", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", - "properties": { - "action": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "button": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "if_present": { - "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", - "type": "boolean" - }, - "kind": { - "const": "alert", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Select a value in a native picker wheel.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "column": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "picker", - "type": "string" - }, - "value": { - "type": "string" - } - }, - "required": [ - "kind", - "value" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", - "properties": { - "action": { - "type": "string" - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": null, - "type": [ - "integer", - "null" - ] - }, - "element": { - "type": "integer" - }, - "kind": { - "const": "perform", - "type": "string" - }, - "snapshot": { - "type": "string" - }, - "value": { - "default": null, - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "element", - "snapshot", - "action" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Poll the current element tree until the semantic expectation holds.", - "properties": { - "expect": { - "$ref": "#/$defs/PhoneUiExpectation" - }, - "kind": { - "const": "wait_for", - "type": "string" - }, - "poll_ms": { - "default": 250, - "type": "integer" - }, - "timeout_ms": { - "default": 15000, - "type": "integer" - } - }, - "required": [ - "kind", - "expect" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "A short animation pause. Prefer `wait_for` for correctness.", - "properties": { - "kind": { - "const": "pause", - "type": "string" - }, - "ms": { - "type": "integer" - } - }, - "required": [ - "kind", - "ms" - ], - "type": "object" - } -]New value: +[ + { + "additionalProperties": false, + "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "tap", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Long-press normalized screen coordinates.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "duration_ms": { + "default": 600, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "longpress", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold 2–5 distinct normalized points simultaneously; array length is finger count.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "duration_ms": { + "default": 1000, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "multi_longpress", + "type": "string" + }, + "points": { + "items": { + "$ref": "#/$defs/PhoneHoldPoint" + }, + "type": "array" + } + }, + "required": [ + "kind", + "points" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Swipe once from one normalized screen point to another.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "swipe", + "type": "string" + }, + "x1": { + "type": "number" + }, + "x2": { + "type": "number" + }, + "y1": { + "type": "number" + }, + "y2": { + "type": "number" + } + }, + "required": [ + "kind", + "x1", + "y1", + "x2", + "y2" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "minimum": 0, + "type": "integer" + }, + "hold_ms": { + "default": 500, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "drag", + "type": "string" + }, + "points": { + "default": null, + "items": { + "$ref": "#/$defs/PhoneDragPoint" + }, + "type": [ + "array", + "null" + ] + }, + "x1": { + "type": [ + "number", + "null" + ] + }, + "x2": { + "type": [ + "number", + "null" + ] + }, + "y1": { + "type": [ + "number", + "null" + ] + }, + "y2": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap one exact, unique accessibility label.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "tap_label", + "type": "string" + }, + "label": { + "type": "string" + } + }, + "required": [ + "kind", + "label" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "if_present": { + "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", + "type": "boolean" + }, + "kind": { + "const": "tap_locator", + "type": "string" + }, + "locator": { + "$ref": "#/$defs/PhoneElementLocator" + }, + "via": { + "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "locator" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "budget_ms": { + "description": "Optional per-call typing budget (6000..15000 ms).", + "maximum": 15000, + "minimum": 6000, + "type": [ + "integer", + "null" + ] + }, + "clear": { + "default": false, + "type": "boolean" + }, + "kind": { + "const": "type", + "type": "string" + }, + "text": { + "type": "string" + } + }, + "required": [ + "kind", + "text" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Continue only the unsent suffix held under a single-use original-field token.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "continue_token": { + "maxLength": 64, + "minLength": 64, + "pattern": "^[0-9a-f]{64}$", + "type": "string" + }, + "kind": { + "const": "type_continue", + "type": "string" + } + }, + "required": [ + "kind", + "continue_token" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Send one supported device-native key.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "key", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Trigger Home or Spotlight.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "shortcut", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "dx": { + "type": "number" + }, + "dy": { + "type": "number" + }, + "kind": { + "const": "scroll", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y", + "dx", + "dy" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; a\nloose name that starts a word of exactly one installed app's name also\nlaunches it, and the step's `resolved` says which; an ambiguous or\nunknown name is refused with candidates, nothing sent).", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "app": { + "type": [ + "string", + "null" + ] + }, + "bundle": { + "type": [ + "string", + "null" + ] + }, + "fresh": { + "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", + "type": "boolean" + }, + "kind": { + "const": "launch_app", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "open_url", + "type": "string" + }, + "url": { + "type": "string" + } + }, + "required": [ + "kind", + "url" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Navigate back inside the current application.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "back", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", + "properties": { + "action": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "button": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "if_present": { + "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", + "type": "boolean" + }, + "kind": { + "const": "alert", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Select a value in a native picker wheel.", + "properties": { + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "column": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "picker", + "type": "string" + }, + "value": { + "type": "string" + } + }, + "required": [ + "kind", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", + "properties": { + "action": { + "type": "string" + }, + "after_ms": { + "default": 0, + "minimum": 0, + "type": "integer" + }, + "duration_ms": { + "default": null, + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "element": { + "minimum": 0, + "type": "integer" + }, + "kind": { + "const": "perform", + "type": "string" + }, + "snapshot": { + "type": "string" + }, + "value": { + "default": null, + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "element", + "snapshot", + "action" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Poll until the expectation holds. For tree waits, timeout_ms:0 makes\none observation attempt bounded to 500ms, without polling or read retry.\nUse observe:false to omit the separate final batch observation.", + "properties": { + "expect": { + "$ref": "#/$defs/PhoneUiExpectation" + }, + "kind": { + "const": "wait_for", + "type": "string" + }, + "poll_ms": { + "default": 250, + "minimum": 0, + "type": "integer" + }, + "timeout_ms": { + "default": 15000, + "minimum": 0, + "type": "integer" + } + }, + "required": [ + "kind", + "expect" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "A short animation pause. Prefer `wait_for` for correctness.", + "properties": { + "kind": { + "const": "pause", + "type": "string" + }, + "ms": { + "minimum": 0, + "type": "integer" + } + }, + "required": [ + "kind", + "ms" + ], + "type": "object" + } +] - changed
Input schema / $defs / PhoneUiExpectation / properties / absent / items / $refPrevious value: -"#/$defs/PhoneElementLocator"New value: +"#/$defs/WaitSelector" - changed
Input schema / $defs / PhoneUiExpectation / properties / present / items / $refPrevious value: -"#/$defs/PhoneElementLocator"New value: +"#/$defs/WaitSelector" - added
Input schema / $defs / WaitIndexAdded value: +{ + "description": "Zero-based ordinal among matches; index 2 requires at least three matches.", + "maximum": 199, + "minimum": 0, + "type": "integer" +} - added
Input schema / $defs / WaitSelectorAdded value: +{ + "additionalProperties": false, + "description": "Flat legacy locator fields plus a wait-only bounded ordinal. Custom parsing\npreserves strict unknown/duplicate-field rejection with flattened schema.", + "properties": { + "enabled": { + "type": [ + "boolean", + "null" + ] + }, + "focused": { + "type": [ + "boolean", + "null" + ] + }, + "identifier": { + "type": [ + "string", + "null" + ] + }, + "index": { + "$ref": "#/$defs/WaitIndex" + }, + "kind": { + "type": [ + "string", + "null" + ] + }, + "label": { + "type": [ + "string", + "null" + ] + }, + "label_contains": { + "description": "Case-sensitive literal substring of the normalized label.", + "maxLength": 1000, + "minLength": 1, + "type": [ + "string", + "null" + ] + }, + "value": { + "type": [ + "string", + "null" + ] + }, + "visible": { + "type": [ + "boolean", + "null" + ] + } + }, + "type": "object" +}
- Changed
phone_screen_frame1 field changed- added
Input schema / properties / max_side / minimumAdded value: +0
- Changed
phone_screenshot1 field changed- added
Input schema / properties / max_side / minimumAdded value: +0
- Changed
phone_scroll_find1 field changed- added
Input schema / properties / max_swipes / minimumAdded value: +0
- Changed
phone_tap_element1 field changed- added
Input schema / properties / element / minimumAdded value: +0
- Changed
phone_type8 fields changed- added
Input schema / properties / budget_msAdded value: +{ + "description": "Optional typing budget, 6000..15000 ms, requiring a named owner and inputContinuation.", + "maximum": 15000, + "minimum": 6000, + "type": [ + "integer", + "null" + ] +} - added
Input schema / properties / clearAdded value: +{ + "description": "Clear the bound field first. Never combine with continue_token.", + "type": [ + "boolean", + "null" + ] +} - added
Input schema / properties / continue_tokenAdded value: +{ + "description": "Single-use token from a safe chunk boundary. Pass without text, clear or budget_ms.", + "maxLength": 64, + "minLength": 64, + "pattern": "^[0-9a-f]{64}$", + "type": [ + "string", + "null" + ] +} - removed
Input schema / properties / observe / defaultRemoved value: -null - changed
Input schema / properties / observe / descriptionPrevious value: -"Default true: return what the screen settled to (as on phone_tap).\n`false` for the fastest bare action."New value: +"Default true: return the settled screen change after a completed action." - changed
Input schema / properties / text / descriptionPrevious value: -"Unicode text to send through the device-side input service. Focus the\nintended field and verify it before typing. Up to 20000 characters;\ntext over 200 is typed in chunks (about a minute per 3000 characters),\nand a failure part-way reports `characters_confirmed` and\n`remaining_text` with retry_safe=false — then type only the rest."New value: +"Unicode text to start typing, up to 20000 scalars. Omit when continuing." - changed
Input schema / properties / text / typePrevious value: -"string"New value: +[ + "string", + "null" +] - removed
Input schema / requiredRemoved value: -[ - "text" -]
2 tool updates
v0.27.21- Changed
phone_elements2 fields changed- added
Input schema / $defsAdded value: +{ + "ElementSelector": { + "additionalProperties": false, + "properties": { + "enabled": { + "description": "Exact enabled state. A missing state never matches either boolean.", + "type": [ + "boolean", + "null" + ] + }, + "identifier": { + "description": "Exact accessibility identifier; case-sensitive.", + "type": [ + "string", + "null" + ] + }, + "kind": { + "description": "Exact accessibility kind, e.g. Button (XCUIElementTypeButton also accepted).", + "type": [ + "string", + "null" + ] + }, + "label": { + "description": "Exact label, preserving case, whitespace and newlines.", + "type": [ + "string", + "null" + ] + }, + "label_contains": { + "description": "Case-sensitive substring of the label. Combined with other fields using AND.", + "type": [ + "string", + "null" + ] + }, + "value": { + "description": "Exact exposed accessibility value; known truncated previews never match.\nAn empty string matches a present empty value.", + "type": [ + "string", + "null" + ] + } + }, + "type": "object" + } +} - added
Input schema / properties / selectorAdded value: +{ + "anyOf": [ + { + "$ref": "#/$defs/ElementSelector" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Literal case-sensitive field filters combined with AND; preserves original snapshot indexes.\nRequires at least one field; filters AX elements, not separate OCR text." +}
- Changed
phone_home_layout3 fields changed- added
Input schema / properties / actionAdded value: +{ + "default": null, + "description": "Omit to read. `backup` saves the current layout on the Mac; `preview`\nshows what writing `file` would change, by app identifier; `apply`\nwrites `file`; `restore` writes the newest automatic backup (or `file`).", + "type": [ + "string", + "null" + ] +} - added
Input schema / properties / confirmAdded value: +{ + "default": false, + "description": "apply/restore only: true writes. Without it the preview comes back and\nnothing changes; show it to the person and ask first.", + "type": "boolean" +} - added
Input schema / properties / fileAdded value: +{ + "default": null, + "description": "Absolute path of a layout .plist on the Mac (from a backup).", + "type": [ + "string", + "null" + ] +}
2 tool updates
v0.27.7- Added
phone_metrics - Changed
phone_shortcut1 field changed- changed
Input schema / properties / name / descriptionPrevious value: -"Supported names: `home` (Home Screen) and `spotlight` (search).\nApp Switcher is unsupported by the device runner."New value: +"Home, Spotlight, Photos capture (`save_screenshot`), or physical-only `volume_up` / `volume_down`.\nApp Switcher is unsupported by the device runner."
40 tool updates
v0.27.3- Changed
phone_app_notes1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_apps1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_capabilities1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_collect_list1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_crashes1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_elements1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_draft1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_info1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_list1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_publish1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_report1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_run1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_flow_update1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_hold1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_home_layout1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_install_app1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Added
phone_instances - Changed
phone_jev_run1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_key1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_login1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_logs1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Added
phone_read_value - Changed
phone_reconnect4 fields changed- added
Input schema / additionalPropertiesAdded value: +false - added
Input schema / descriptionAdded value: +"Parameters for [`PhoneHandler::phone_reconnect`]." - added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +} - added
Input schema / properties / wait_secsAdded value: +{ + "default": null, + "description": "Wait up to this many seconds (1–300) for the outcome instead of\nreturning at once: the call returns when the phone is drivable, a\nblocker appears (setup_blocked_on, e.g. the passcode), or the time is\nup — with the compact final state (a blocker other than the passcode\nmust hold for 20 s first). A runner rebuild after an upgrade\ntakes 1–2 min, so 180 covers it. Omit to return immediately.", + "type": [ + "integer", + "null" + ] +}
- Changed
phone_record1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_release_owner1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_run_end1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_run_start1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_run_steps4 fields changed- changed
Input schema / $defs / PhoneStep / oneOfPrevious value: -[ - { - "additionalProperties": false, - "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Long-press normalized screen coordinates.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 600, - "type": "integer" - }, - "kind": { - "const": "longpress", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold 2–5 distinct normalized points simultaneously; array length is finger count.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 1000, - "type": "integer" - }, - "kind": { - "const": "multi_longpress", - "type": "string" - }, - "points": { - "items": { - "$ref": "#/$defs/PhoneHoldPoint" - }, - "type": "array" - } - }, - "required": [ - "kind", - "points" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Swipe once from one normalized screen point to another.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "kind": { - "const": "swipe", - "type": "string" - }, - "x1": { - "type": "number" - }, - "x2": { - "type": "number" - }, - "y1": { - "type": "number" - }, - "y2": { - "type": "number" - } - }, - "required": [ - "kind", - "x1", - "y1", - "x2", - "y2" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "hold_ms": { - "default": 500, - "type": "integer" - }, - "kind": { - "const": "drag", - "type": "string" - }, - "points": { - "default": null, - "items": { - "$ref": "#/$defs/PhoneDragPoint" - }, - "type": [ - "array", - "null" - ] - }, - "x1": { - "type": [ - "number", - "null" - ] - }, - "x2": { - "type": [ - "number", - "null" - ] - }, - "y1": { - "type": [ - "number", - "null" - ] - }, - "y2": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap one exact, unique accessibility label.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap_label", - "type": "string" - }, - "label": { - "type": "string" - } - }, - "required": [ - "kind", - "label" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "if_present": { - "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", - "type": "boolean" - }, - "kind": { - "const": "tap_locator", - "type": "string" - }, - "locator": { - "$ref": "#/$defs/PhoneElementLocator" - }, - "via": { - "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "locator" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "clear": { - "default": false, - "type": "boolean" - }, - "kind": { - "const": "type", - "type": "string" - }, - "text": { - "type": "string" - } - }, - "required": [ - "kind", - "text" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Send one supported device-native key.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "key", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Trigger Home or Spotlight.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "shortcut", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "dx": { - "type": "number" - }, - "dy": { - "type": "number" - }, - "kind": { - "const": "scroll", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y", - "dx", - "dy" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its exact name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; an\nambiguous or unknown name is refused with candidates, nothing sent).", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "app": { - "type": [ - "string", - "null" - ] - }, - "bundle": { - "type": [ - "string", - "null" - ] - }, - "fresh": { - "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", - "type": "boolean" - }, - "kind": { - "const": "launch_app", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "open_url", - "type": "string" - }, - "url": { - "type": "string" - } - }, - "required": [ - "kind", - "url" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Navigate back inside the current application.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "back", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", - "properties": { - "action": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "button": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "if_present": { - "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", - "type": "boolean" - }, - "kind": { - "const": "alert", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Select a value in a native picker wheel.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "column": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "picker", - "type": "string" - }, - "value": { - "type": "string" - } - }, - "required": [ - "kind", - "value" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", - "properties": { - "action": { - "type": "string" - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": null, - "type": [ - "integer", - "null" - ] - }, - "element": { - "type": "integer" - }, - "kind": { - "const": "perform", - "type": "string" - }, - "snapshot": { - "type": "string" - }, - "value": { - "default": null, - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "element", - "snapshot", - "action" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Poll the current element tree until the semantic expectation holds.", - "properties": { - "expect": { - "$ref": "#/$defs/PhoneUiExpectation" - }, - "kind": { - "const": "wait_for", - "type": "string" - }, - "poll_ms": { - "default": 250, - "type": "integer" - }, - "timeout_ms": { - "default": 15000, - "type": "integer" - } - }, - "required": [ - "kind", - "expect" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "A short animation pause. Prefer `wait_for` for correctness.", - "properties": { - "kind": { - "const": "pause", - "type": "string" - }, - "ms": { - "type": "integer" - } - }, - "required": [ - "kind", - "ms" - ], - "type": "object" - } -]New value: +[ + { + "additionalProperties": false, + "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Long-press normalized screen coordinates.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 600, + "type": "integer" + }, + "kind": { + "const": "longpress", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold 2–5 distinct normalized points simultaneously; array length is finger count.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 1000, + "type": "integer" + }, + "kind": { + "const": "multi_longpress", + "type": "string" + }, + "points": { + "items": { + "$ref": "#/$defs/PhoneHoldPoint" + }, + "type": "array" + } + }, + "required": [ + "kind", + "points" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Swipe once from one normalized screen point to another.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "kind": { + "const": "swipe", + "type": "string" + }, + "x1": { + "type": "number" + }, + "x2": { + "type": "number" + }, + "y1": { + "type": "number" + }, + "y2": { + "type": "number" + } + }, + "required": [ + "kind", + "x1", + "y1", + "x2", + "y2" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "hold_ms": { + "default": 500, + "type": "integer" + }, + "kind": { + "const": "drag", + "type": "string" + }, + "points": { + "default": null, + "items": { + "$ref": "#/$defs/PhoneDragPoint" + }, + "type": [ + "array", + "null" + ] + }, + "x1": { + "type": [ + "number", + "null" + ] + }, + "x2": { + "type": [ + "number", + "null" + ] + }, + "y1": { + "type": [ + "number", + "null" + ] + }, + "y2": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap one exact, unique accessibility label.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap_label", + "type": "string" + }, + "label": { + "type": "string" + } + }, + "required": [ + "kind", + "label" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "if_present": { + "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", + "type": "boolean" + }, + "kind": { + "const": "tap_locator", + "type": "string" + }, + "locator": { + "$ref": "#/$defs/PhoneElementLocator" + }, + "via": { + "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "locator" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "clear": { + "default": false, + "type": "boolean" + }, + "kind": { + "const": "type", + "type": "string" + }, + "text": { + "type": "string" + } + }, + "required": [ + "kind", + "text" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Send one supported device-native key.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "key", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Trigger Home or Spotlight.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "shortcut", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "dx": { + "type": "number" + }, + "dy": { + "type": "number" + }, + "kind": { + "const": "scroll", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y", + "dx", + "dy" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; a\nloose name that starts a word of exactly one installed app's name also\nlaunches it, and the step's `resolved` says which; an ambiguous or\nunknown name is refused with candidates, nothing sent).", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "app": { + "type": [ + "string", + "null" + ] + }, + "bundle": { + "type": [ + "string", + "null" + ] + }, + "fresh": { + "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", + "type": "boolean" + }, + "kind": { + "const": "launch_app", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "open_url", + "type": "string" + }, + "url": { + "type": "string" + } + }, + "required": [ + "kind", + "url" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Navigate back inside the current application.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "back", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", + "properties": { + "action": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "button": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "if_present": { + "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", + "type": "boolean" + }, + "kind": { + "const": "alert", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Select a value in a native picker wheel.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "column": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "picker", + "type": "string" + }, + "value": { + "type": "string" + } + }, + "required": [ + "kind", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", + "properties": { + "action": { + "type": "string" + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": null, + "type": [ + "integer", + "null" + ] + }, + "element": { + "type": "integer" + }, + "kind": { + "const": "perform", + "type": "string" + }, + "snapshot": { + "type": "string" + }, + "value": { + "default": null, + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "element", + "snapshot", + "action" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Poll the current element tree until the semantic expectation holds.", + "properties": { + "expect": { + "$ref": "#/$defs/PhoneUiExpectation" + }, + "kind": { + "const": "wait_for", + "type": "string" + }, + "poll_ms": { + "default": 250, + "type": "integer" + }, + "timeout_ms": { + "default": 15000, + "type": "integer" + } + }, + "required": [ + "kind", + "expect" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "A short animation pause. Prefer `wait_for` for correctness.", + "properties": { + "kind": { + "const": "pause", + "type": "string" + }, + "ms": { + "type": "integer" + } + }, + "required": [ + "kind", + "ms" + ], + "type": "object" + } +] - added
Input schema / $defs / PhoneUiExpectation / properties / ocr_absentAdded value: +{ + "description": "Text that must not be visible (a loading caption), read by OCR.", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / $defs / PhoneUiExpectation / properties / ocr_presentAdded value: +{ + "description": "Text that must be visible, read by OCR from screenshots — for canvas\nscreens (games, Flutter/Unity) whose element tree has no labels, e.g.\nwaiting out a game's loading screen. Case and spaces are ignored.\nCannot be combined with application/present/absent in one wait_for;\nsuch a wait may take timeout_ms up to 60000.", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_screen1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_screen_frame1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_screenshot1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_scroll1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_scroll_find1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_set_location1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_shortcut1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_status1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_tap1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_tap_element1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_tap_label1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
- Changed
phone_type1 field changed- added
Input schema / properties / instanceAdded value: +{ + "description": "Installed local instance name, such as i13 or default. Omit to use the configured daemon. No fallback on failure.", + "type": "string" +}
1 tool update
v0.23.0- Changed
phone_run_steps2 fields changed- added
Input schema / $defs / PhoneHoldPointAdded value: +{ + "additionalProperties": false, + "description": "One independently placed finger in a simultaneous hold.", + "properties": { + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y" + ], + "type": "object" +} - changed
Input schema / $defs / PhoneStep / oneOfPrevious value: -[ - { - "additionalProperties": false, - "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Long-press normalized screen coordinates.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 600, - "type": "integer" - }, - "kind": { - "const": "longpress", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Swipe once from one normalized screen point to another.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "kind": { - "const": "swipe", - "type": "string" - }, - "x1": { - "type": "number" - }, - "x2": { - "type": "number" - }, - "y1": { - "type": "number" - }, - "y2": { - "type": "number" - } - }, - "required": [ - "kind", - "x1", - "y1", - "x2", - "y2" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "hold_ms": { - "default": 500, - "type": "integer" - }, - "kind": { - "const": "drag", - "type": "string" - }, - "points": { - "default": null, - "items": { - "$ref": "#/$defs/PhoneDragPoint" - }, - "type": [ - "array", - "null" - ] - }, - "x1": { - "type": [ - "number", - "null" - ] - }, - "x2": { - "type": [ - "number", - "null" - ] - }, - "y1": { - "type": [ - "number", - "null" - ] - }, - "y2": { - "type": [ - "number", - "null" - ] - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap one exact, unique accessibility label.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap_label", - "type": "string" - }, - "label": { - "type": "string" - } - }, - "required": [ - "kind", - "label" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "if_present": { - "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", - "type": "boolean" - }, - "kind": { - "const": "tap_locator", - "type": "string" - }, - "locator": { - "$ref": "#/$defs/PhoneElementLocator" - }, - "via": { - "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "locator" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "clear": { - "default": false, - "type": "boolean" - }, - "kind": { - "const": "type", - "type": "string" - }, - "text": { - "type": "string" - } - }, - "required": [ - "kind", - "text" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Send one supported device-native key.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "key", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Trigger Home or Spotlight.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "shortcut", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "dx": { - "type": "number" - }, - "dy": { - "type": "number" - }, - "kind": { - "const": "scroll", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y", - "dx", - "dy" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its exact name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; an\nambiguous or unknown name is refused with candidates, nothing sent).", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "app": { - "type": [ - "string", - "null" - ] - }, - "bundle": { - "type": [ - "string", - "null" - ] - }, - "fresh": { - "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", - "type": "boolean" - }, - "kind": { - "const": "launch_app", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "open_url", - "type": "string" - }, - "url": { - "type": "string" - } - }, - "required": [ - "kind", - "url" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Navigate back inside the current application.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "back", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", - "properties": { - "action": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "button": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "if_present": { - "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", - "type": "boolean" - }, - "kind": { - "const": "alert", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Select a value in a native picker wheel.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "column": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "picker", - "type": "string" - }, - "value": { - "type": "string" - } - }, - "required": [ - "kind", - "value" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", - "properties": { - "action": { - "type": "string" - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": null, - "type": [ - "integer", - "null" - ] - }, - "element": { - "type": "integer" - }, - "kind": { - "const": "perform", - "type": "string" - }, - "snapshot": { - "type": "string" - }, - "value": { - "default": null, - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "element", - "snapshot", - "action" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Poll the current element tree until the semantic expectation holds.", - "properties": { - "expect": { - "$ref": "#/$defs/PhoneUiExpectation" - }, - "kind": { - "const": "wait_for", - "type": "string" - }, - "poll_ms": { - "default": 250, - "type": "integer" - }, - "timeout_ms": { - "default": 15000, - "type": "integer" - } - }, - "required": [ - "kind", - "expect" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "A short animation pause. Prefer `wait_for` for correctness.", - "properties": { - "kind": { - "const": "pause", - "type": "string" - }, - "ms": { - "type": "integer" - } - }, - "required": [ - "kind", - "ms" - ], - "type": "object" - } -]New value: +[ + { + "additionalProperties": false, + "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Long-press normalized screen coordinates.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 600, + "type": "integer" + }, + "kind": { + "const": "longpress", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold 2–5 distinct normalized points simultaneously; array length is finger count.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 1000, + "type": "integer" + }, + "kind": { + "const": "multi_longpress", + "type": "string" + }, + "points": { + "items": { + "$ref": "#/$defs/PhoneHoldPoint" + }, + "type": "array" + } + }, + "required": [ + "kind", + "points" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Swipe once from one normalized screen point to another.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "kind": { + "const": "swipe", + "type": "string" + }, + "x1": { + "type": "number" + }, + "x2": { + "type": "number" + }, + "y1": { + "type": "number" + }, + "y2": { + "type": "number" + } + }, + "required": [ + "kind", + "x1", + "y1", + "x2", + "y2" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "hold_ms": { + "default": 500, + "type": "integer" + }, + "kind": { + "const": "drag", + "type": "string" + }, + "points": { + "default": null, + "items": { + "$ref": "#/$defs/PhoneDragPoint" + }, + "type": [ + "array", + "null" + ] + }, + "x1": { + "type": [ + "number", + "null" + ] + }, + "x2": { + "type": [ + "number", + "null" + ] + }, + "y1": { + "type": [ + "number", + "null" + ] + }, + "y2": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap one exact, unique accessibility label.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap_label", + "type": "string" + }, + "label": { + "type": "string" + } + }, + "required": [ + "kind", + "label" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "if_present": { + "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", + "type": "boolean" + }, + "kind": { + "const": "tap_locator", + "type": "string" + }, + "locator": { + "$ref": "#/$defs/PhoneElementLocator" + }, + "via": { + "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "locator" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "clear": { + "default": false, + "type": "boolean" + }, + "kind": { + "const": "type", + "type": "string" + }, + "text": { + "type": "string" + } + }, + "required": [ + "kind", + "text" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Send one supported device-native key.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "key", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Trigger Home or Spotlight.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "shortcut", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "dx": { + "type": "number" + }, + "dy": { + "type": "number" + }, + "kind": { + "const": "scroll", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y", + "dx", + "dy" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its exact name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; an\nambiguous or unknown name is refused with candidates, nothing sent).", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "app": { + "type": [ + "string", + "null" + ] + }, + "bundle": { + "type": [ + "string", + "null" + ] + }, + "fresh": { + "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", + "type": "boolean" + }, + "kind": { + "const": "launch_app", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "open_url", + "type": "string" + }, + "url": { + "type": "string" + } + }, + "required": [ + "kind", + "url" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Navigate back inside the current application.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "back", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", + "properties": { + "action": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "button": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "if_present": { + "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", + "type": "boolean" + }, + "kind": { + "const": "alert", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Select a value in a native picker wheel.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "column": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "picker", + "type": "string" + }, + "value": { + "type": "string" + } + }, + "required": [ + "kind", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", + "properties": { + "action": { + "type": "string" + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": null, + "type": [ + "integer", + "null" + ] + }, + "element": { + "type": "integer" + }, + "kind": { + "const": "perform", + "type": "string" + }, + "snapshot": { + "type": "string" + }, + "value": { + "default": null, + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "element", + "snapshot", + "action" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Poll the current element tree until the semantic expectation holds.", + "properties": { + "expect": { + "$ref": "#/$defs/PhoneUiExpectation" + }, + "kind": { + "const": "wait_for", + "type": "string" + }, + "poll_ms": { + "default": 250, + "type": "integer" + }, + "timeout_ms": { + "default": 15000, + "type": "integer" + } + }, + "required": [ + "kind", + "expect" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "A short animation pause. Prefer `wait_for` for correctness.", + "properties": { + "kind": { + "const": "pause", + "type": "string" + }, + "ms": { + "type": "integer" + } + }, + "required": [ + "kind", + "ms" + ], + "type": "object" + } +]
3 tool updates
v0.22.4-glama.1- Changed
phone_scroll4 fields changed- added
Input schema / properties / x / maximumAdded value: +1 - added
Input schema / properties / x / minimumAdded value: +0 - added
Input schema / properties / y / maximumAdded value: +1 - added
Input schema / properties / y / minimumAdded value: +0
- Changed
phone_tap4 fields changed- added
Input schema / properties / x / maximumAdded value: +1 - added
Input schema / properties / x / minimumAdded value: +0 - added
Input schema / properties / y / maximumAdded value: +1 - added
Input schema / properties / y / minimumAdded value: +0
- Changed
phone_type1 field changed- added
Input schema / properties / text / maxLengthAdded value: +20000
2 tool updates
v0.22.3- Added
phone_home_layout - Changed
phone_run_steps2 fields changed- added
Input schema / $defs / PhoneDragPointAdded value: +{ + "additionalProperties": false, + "description": "One stop in a continuous drag. Dwell keeps the same finger down.", + "properties": { + "dwell_ms": { + "default": 0, + "type": "integer" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "x", + "y" + ], + "type": "object" +} - changed
Input schema / $defs / PhoneStep / oneOfPrevious value: -[ - { - "additionalProperties": false, - "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Long-press normalized screen coordinates.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 600, - "type": "integer" - }, - "kind": { - "const": "longpress", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Swipe once from one normalized screen point to another.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "kind": { - "const": "swipe", - "type": "string" - }, - "x1": { - "type": "number" - }, - "x2": { - "type": "number" - }, - "y1": { - "type": "number" - }, - "y2": { - "type": "number" - } - }, - "required": [ - "kind", - "x1", - "y1", - "x2", - "y2" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Hold, then drag once between two normalized screen points.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": 300, - "type": "integer" - }, - "hold_ms": { - "default": 500, - "type": "integer" - }, - "kind": { - "const": "drag", - "type": "string" - }, - "x1": { - "type": "number" - }, - "x2": { - "type": "number" - }, - "y1": { - "type": "number" - }, - "y2": { - "type": "number" - } - }, - "required": [ - "kind", - "x1", - "y1", - "x2", - "y2" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap one exact, unique accessibility label.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "tap_label", - "type": "string" - }, - "label": { - "type": "string" - } - }, - "required": [ - "kind", - "label" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "if_present": { - "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", - "type": "boolean" - }, - "kind": { - "const": "tap_locator", - "type": "string" - }, - "locator": { - "$ref": "#/$defs/PhoneElementLocator" - }, - "via": { - "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "locator" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "clear": { - "default": false, - "type": "boolean" - }, - "kind": { - "const": "type", - "type": "string" - }, - "text": { - "type": "string" - } - }, - "required": [ - "kind", - "text" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Send one supported device-native key.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "key", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Trigger Home or Spotlight.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "shortcut", - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": [ - "kind", - "name" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "dx": { - "type": "number" - }, - "dy": { - "type": "number" - }, - "kind": { - "const": "scroll", - "type": "string" - }, - "x": { - "type": "number" - }, - "y": { - "type": "number" - } - }, - "required": [ - "kind", - "x", - "y", - "dx", - "dy" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its exact name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; an\nambiguous or unknown name is refused with candidates, nothing sent).", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "app": { - "type": [ - "string", - "null" - ] - }, - "bundle": { - "type": [ - "string", - "null" - ] - }, - "fresh": { - "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", - "type": "boolean" - }, - "kind": { - "const": "launch_app", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "open_url", - "type": "string" - }, - "url": { - "type": "string" - } - }, - "required": [ - "kind", - "url" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Navigate back inside the current application.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "back", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", - "properties": { - "action": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "button": { - "default": null, - "type": [ - "string", - "null" - ] - }, - "if_present": { - "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", - "type": "boolean" - }, - "kind": { - "const": "alert", - "type": "string" - } - }, - "required": [ - "kind" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Select a value in a native picker wheel.", - "properties": { - "after_ms": { - "default": 0, - "type": "integer" - }, - "column": { - "default": 0, - "type": "integer" - }, - "kind": { - "const": "picker", - "type": "string" - }, - "value": { - "type": "string" - } - }, - "required": [ - "kind", - "value" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", - "properties": { - "action": { - "type": "string" - }, - "after_ms": { - "default": 0, - "type": "integer" - }, - "duration_ms": { - "default": null, - "type": [ - "integer", - "null" - ] - }, - "element": { - "type": "integer" - }, - "kind": { - "const": "perform", - "type": "string" - }, - "snapshot": { - "type": "string" - }, - "value": { - "default": null, - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "kind", - "element", - "snapshot", - "action" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "Poll the current element tree until the semantic expectation holds.", - "properties": { - "expect": { - "$ref": "#/$defs/PhoneUiExpectation" - }, - "kind": { - "const": "wait_for", - "type": "string" - }, - "poll_ms": { - "default": 250, - "type": "integer" - }, - "timeout_ms": { - "default": 15000, - "type": "integer" - } - }, - "required": [ - "kind", - "expect" - ], - "type": "object" - }, - { - "additionalProperties": false, - "description": "A short animation pause. Prefer `wait_for` for correctness.", - "properties": { - "kind": { - "const": "pause", - "type": "string" - }, - "ms": { - "type": "integer" - } - }, - "required": [ - "kind", - "ms" - ], - "type": "object" - } -]New value: +[ + { + "additionalProperties": false, + "description": "Tap normalized screen coordinates. Prefer `tap_locator` (durable in a\nsaved flow) whenever the target has a label or identifier.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Long-press normalized screen coordinates.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 600, + "type": "integer" + }, + "kind": { + "const": "longpress", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Swipe once from one normalized screen point to another.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "kind": { + "const": "swipe", + "type": "string" + }, + "x1": { + "type": "number" + }, + "x2": { + "type": "number" + }, + "y1": { + "type": "number" + }, + "y2": { + "type": "number" + } + }, + "required": [ + "kind", + "x1", + "y1", + "x2", + "y2" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Hold, then drag continuously. Use either x1/y1/x2/y2 or 2–16\nnormalized points with optional dwell_ms at each stop. For points,\nduration_ms is travel time per segment; total hold/travel/dwell <= 10s.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": 300, + "type": "integer" + }, + "hold_ms": { + "default": 500, + "type": "integer" + }, + "kind": { + "const": "drag", + "type": "string" + }, + "points": { + "default": null, + "items": { + "$ref": "#/$defs/PhoneDragPoint" + }, + "type": [ + "array", + "null" + ] + }, + "x1": { + "type": [ + "number", + "null" + ] + }, + "x2": { + "type": [ + "number", + "null" + ] + }, + "y1": { + "type": [ + "number", + "null" + ] + }, + "y2": { + "type": [ + "number", + "null" + ] + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap one exact, unique accessibility label.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "tap_label", + "type": "string" + }, + "label": { + "type": "string" + } + }, + "required": [ + "kind", + "label" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Tap the one current element matching every supplied semantic locator\nfield. Zero or multiple matches send no tap.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "if_present": { + "description": "Pass instead of failing when no element matches — for screens that\nappear only sometimes (\"Not now\", \"Skip\"). Nothing is sent then.", + "type": "boolean" + }, + "kind": { + "const": "tap_locator", + "type": "string" + }, + "locator": { + "$ref": "#/$defs/PhoneElementLocator" + }, + "via": { + "description": "`point`: tap the centre of the element's live frame instead of\nXCUIElement's click — for custom controls that ACK a click and do\nnothing. Default `element`.", + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "locator" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Type Unicode text into the focused field. `clear=true` clears that field\nimmediately before inserting the text as one compound action.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "clear": { + "default": false, + "type": "boolean" + }, + "kind": { + "const": "type", + "type": "string" + }, + "text": { + "type": "string" + } + }, + "required": [ + "kind", + "text" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Send one supported device-native key.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "key", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Trigger Home or Spotlight.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "shortcut", + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": [ + "kind", + "name" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Scroll with the same normalized anchor and deltas as `phone_scroll`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "dx": { + "type": "number" + }, + "dy": { + "type": "number" + }, + "kind": { + "const": "scroll", + "type": "string" + }, + "x": { + "type": "number" + }, + "y": { + "type": "number" + } + }, + "required": [ + "kind", + "x", + "y", + "dx", + "dy" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Launch or foreground an installed app: give exactly one of `bundle`\n(its exact bundle identifier) or `app` (its exact name — 微信, WeChat,\n招商银行, Settings — which the daemon resolves through phone_apps; an\nambiguous or unknown name is refused with candidates, nothing sent).", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "app": { + "type": [ + "string", + "null" + ] + }, + "bundle": { + "type": [ + "string", + "null" + ] + }, + "fresh": { + "description": "Quit the app first so it opens on its first screen, not the page\nit was left on (flows recorded on the phone start there).", + "type": "boolean" + }, + "kind": { + "const": "launch_app", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Open a link (`https://…`, or a deep link such as `weixin://`) with\nthe app iOS assigns to it. http/https always work; other schemes\n(tel, sms, mailto, app deep links) are refused unless the operator\nallowed them in IPHONE_USE_ALLOW_URL_SCHEMES. It leaves the current\napp, so guard the next screen with `wait_for`.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "open_url", + "type": "string" + }, + "url": { + "type": "string" + } + }, + "required": [ + "kind", + "url" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Navigate back inside the current application.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "back", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Press a button on a system alert (UIAlertController) through the device runner's\nnative alert route — the only path that actually acts on one; taps on\nalert buttons are acknowledged without effect. Give the exact button\ntext, or `action: accept|dismiss` for the default/cancel button. Fails\nclosed when no alert is showing.", + "properties": { + "action": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "button": { + "default": null, + "type": [ + "string", + "null" + ] + }, + "if_present": { + "description": "Pass instead of failing when no alert is up — for prompts that\nappear only sometimes (\"Allow Paste\"). Flows using it need a\nclient that knows it.", + "type": "boolean" + }, + "kind": { + "const": "alert", + "type": "string" + } + }, + "required": [ + "kind" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Select a value in a native picker wheel.", + "properties": { + "after_ms": { + "default": 0, + "type": "integer" + }, + "column": { + "default": 0, + "type": "integer" + }, + "kind": { + "const": "picker", + "type": "string" + }, + "value": { + "type": "string" + } + }, + "required": [ + "kind", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Invoke a named affordance on a snapshot-bound element: toggle a\nswitch, increment/decrement or adjust a slider or stepper, and the\nother names in phone_capabilities (`perform_actions`). `element` and\n`snapshot` come from phone_elements. Snapshot-bound, so not for saved\nflows. `value` only with `adjust`; `duration_ms` only with\n`menu`/`force_press`. Verify the new `value` with wait_for.", + "properties": { + "action": { + "type": "string" + }, + "after_ms": { + "default": 0, + "type": "integer" + }, + "duration_ms": { + "default": null, + "type": [ + "integer", + "null" + ] + }, + "element": { + "type": "integer" + }, + "kind": { + "const": "perform", + "type": "string" + }, + "snapshot": { + "type": "string" + }, + "value": { + "default": null, + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "kind", + "element", + "snapshot", + "action" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "Poll the current element tree until the semantic expectation holds.", + "properties": { + "expect": { + "$ref": "#/$defs/PhoneUiExpectation" + }, + "kind": { + "const": "wait_for", + "type": "string" + }, + "poll_ms": { + "default": 250, + "type": "integer" + }, + "timeout_ms": { + "default": 15000, + "type": "integer" + } + }, + "required": [ + "kind", + "expect" + ], + "type": "object" + }, + { + "additionalProperties": false, + "description": "A short animation pause. Prefer `wait_for` for correctness.", + "properties": { + "kind": { + "const": "pause", + "type": "string" + }, + "ms": { + "type": "integer" + } + }, + "required": [ + "kind", + "ms" + ], + "type": "object" + } +]
37 tool updates
v0.21.0- First observed
phone_app_notes - First observed
phone_apps - First observed
phone_capabilities - First observed
phone_collect_list - First observed
phone_crashes - First observed
phone_elements - First observed
phone_flow_draft - First observed
phone_flow_info - First observed
phone_flow_list - First observed
phone_flow_publish - First observed
phone_flow_report - First observed
phone_flow_run - First observed
phone_flow_update - First observed
phone_hold - First observed
phone_install_app - First observed
phone_jev_run - First observed
phone_key - First observed
phone_login - First observed
phone_logs - First observed
phone_reconnect - First observed
phone_record - First observed
phone_release_owner - First observed
phone_run_end - First observed
phone_run_start - First observed
phone_run_steps - First observed
phone_screen - First observed
phone_screen_frame - First observed
phone_screenshot - First observed
phone_scroll - First observed
phone_scroll_find - First observed
phone_set_location - First observed
phone_shortcut - First observed
phone_status - First observed
phone_tap - First observed
phone_tap_element - First observed
phone_tap_label - First observed
phone_type
TDQS
Scored across 41 tools
Most tools target clearly distinct capabilities, and the detailed descriptions explicitly differentiate close variants such as phone_tap, phone_tap_element, and phone_tap_label, or phone_screen, phone_screenshot, and phone_screen_frame. A few overlapping clusters remain—tap/scroll/observation/flow families—but their boundaries are largely well documented.
All tools use the predictable phone_ prefix and snake_case, making the set easy to scan. The pattern is mostly consistent, though it mixes verb-first names (phone_tap, phone_install_app) with noun-style read tools (phone_status, phone_elements) and one branded outlier (phone_jev_run).
With 41 tools, the surface is heavy for an agent to navigate, especially because several are niche administrative or internal tools. While the broad iPhone-automation domain justifies many capabilities, this count is well above a well-scoped set and increases selection cost.
The set covers a remarkably full lifecycle: app discovery, UI inspection, tapping, typing, scrolling, flows, installation, login, recording, diagnostics, location simulation, home layout, ownership, and metrics. Minor gaps exist—for example, no standalone direct launch/uninstall or clipboard/notification operations—but they are largely workable through phone_run_steps and related tools.
Maintenance
Related MCP Connectors
Let your AI agent control real iPhones: screenshots, tap, swipe, type and open apps.
Give an AI agent an iPhone to see, tap and type on over MCP. Free early access (Simulator).
Let AI agents control Android and iOS devices on Mobilerun Cloud: screen, input, apps, workflows.
Cloud iPhone Simulators for AI agents: build the Xcode project, read the screen, tap by label, test.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to control real iPhones and simulators on macOS, allowing for UI interaction, testing, and automation.58 npm92MIT
- AlicenseAqualityAmaintenanceEnables AI agents to drive an iPhone or iOS Simulator through 30 tools and 4 resources, using an observe/act/verify loop with built-in safety approvals.3141 PyPI3MIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to operate a real iPhone from a Mac by reading the screen over USB and sending taps, swipes, typing, and app launches over Bluetooth, with every action returning a fresh screenshot so the agent can verify its work. Runs as a local MCP server, so agents can test iOS apps on physical devices and automate everyday phone tasks.43 npm10MIT

Phone Harness MCPofficial
AlicenseAqualityBmaintenanceLets AI agents drive real iPhones and Androids hosted in the cloud over HTTPS — taking screenshots, inspecting on-screen elements, tapping, typing, swiping, and launching apps — with no pairing, adb, or Mac required. It can run locally over stdio or as a hosted streamable HTTP endpoint supporting API keys and OAuth sign-in.15MIT