Skip to main content
Glama

ghosthand

Natural-language automation for iOS: an agent (Claude) drives an iOS Simulator or a real iPhone the way a person would.

  • Runner/ (Swift): an XCUITest app that runs on the device and serves a small HTTP API: accessibility tree, screenshots, taps, swipes, typing, buttons, app launching.

  • ghosthand/ (Python): drives simulators (simctl) and iPhones (devicectl), starts and talks to the runner, exposes everything as an MCP server, and includes an autonomous agent (Claude via the Anthropic API) and a web console.

  • packages/ghosthand-client/: zero-dependency Python SDK to drive the console from pytest or CI.

Requirements

  • macOS with Xcode (tested with Xcode 27.1) and an iOS Simulator runtime (xcodebuild -downloadPlatform iOS)

  • uv (brew install uv)

Related MCP server: ios-agent-driver

Getting started

uv sync
uv run ghosthand boot "iPhone 17"     # boot a simulator
uv run ghosthand runner start         # build (first time) and start the runner
uv run ghosthand tree                 # visible elements, numbered
uv run ghosthand tap -i 4             # tap element [4]
uv run ghosthand launch com.apple.Preferences
uv run ghosthand swipe up
uv run ghosthand type "hello"
uv run ghosthand screenshot screen.png
uv run ghosthand doctor

Default target: the only booted simulator, else the only reachable iPhone. Otherwise pick one with --device "iPhone 17" or GHOSTHAND_DEVICE. The runner is rebuilt automatically when its Swift sources change. Its logs are in ~/.ghosthand/runners/.

On a real iPhone

One-time setup:

  1. Xcode > Settings > Accounts: add your Apple ID. A free account works ("Personal Team").

  2. iPhone: connect it over USB and accept "Trust This Computer". Afterwards, Wi-Fi works too.

  3. Developer Mode: Settings > Privacy & Security > Developer Mode (requires a restart).

  4. Auto-Lock: Settings > Display & Brightness > Auto-Lock > Never while automating. A locked iPhone cannot be driven; the agent stops with a clear message.

uv run ghosthand devices                          # the iPhone must be "reachable"
uv run ghosthand runner start -d "My iPhone"      # builds, signs, installs and starts the runner

On the very first launch, iOS refuses apps from an unknown developer: on the iPhone, go to Settings > General > VPN & Device Management > Apple Development: > Trust, then start again. The CLI prints these instructions when it detects the problem.

How it works:

  • the runner is signed with your team (set GHOSTHAND_TEAM to force one) and bundle IDs unique to your account;

  • with a free account, signing expires after 7 days: the runner is rebuilt automatically after 6 days;

  • the Mac reaches the runner through the CoreDevice tunnel that Xcode opens to the iPhone (USB or Wi-Fi). Its IPv6 address is resolved once at startup, because each mDNS lookup of <udid>.coredevice.local takes several seconds;

  • screenshots are downscaled to point size and JPEG-encoded on the iPhone: about 30 KB instead of 1.7 MB.

With Claude Code (MCP)

uv run ghosthand mcp install            # = claude mcp add --scope user ghosthand -- uv --directory <here> run ghosthand mcp serve

Restart Claude Code, then ask for example: "On the simulator, open Settings and turn on dark mode." The runner starts on the first call. No API key is needed: Claude Code does the thinking.

Tool

Purpose

ios_observe

numbered element list + annotated screenshot (scaled to points)

ios_tap

tap by index or x/y, long press with hold_seconds

ios_swipe

a direction, or start and end points

ios_type

type into the focused field

ios_press

Home button

ios_launch / ios_terminate

open / quit an app

ios_open_url

URL or deep link

ios_apps

installed apps

ios_devices / ios_boot

list targets / get one ready (and start its runner)

ios_run_task / ios_task_status / ios_task_cancel

hand a whole task to the built-in agent

Each action returns the refreshed element list, saving a round trip. An index always refers to the latest observation returned to the model.

Autonomous agent

export ANTHROPIC_API_KEY=sk-ant-...        # or `ant auth login`
uv run ghosthand run "Turn on dark mode in Settings"
uv run ghosthand traces                  # recent runs

Each step is one independent request to the model: the task, a text journal of previous steps and the current screen (numbered elements + annotated screenshot). The model answers with exactly one action (tap_element, tap_point, long_press_element, swipe, swipe_points, type_text, press_home, launch_app, open_url, wait or finish) and a one-sentence reason.

  • Old screenshots are never resent: only the text journal grows, so the cost per step stays flat.

  • An action with no visible effect is reported to the model ("the screen did not change"), and so are errors.

  • Stops on finish, the step budget (--max-steps, default 30), a model refusal, or 3 answers without an action.

  • Model: claude-opus-5-5 at low effort by default (--model, --effort, or GHOSTHAND_MODEL / GHOSTHAND_EFFORT). Server-side refusal fallbacks (fallbacks: "default") are enabled.

Reliable mode

uv run ghosthand run "Turn on dark mode in Settings" --mode reliable
  • Plan: before acting, one request produces 2 to 6 checkpoints, each with the evidence expected on screen.

  • Verification: the agent claims a checkpoint with checkpoint_done; an independent reviewer, which took no action and sees only the screen and the journal, confirms or rejects it. finish(success=true) is reviewed the same way: after two rejections the task fails instead of reporting a false success.

  • Replanning: replan when the situation no longer matches the plan.

  • Slower and more expensive than fast (one extra request per checkpoint), but a reported success is a seen success.

There is no risk guard: the prompts tell the model not to complete purchases, deletions or messages unless the task asks for them, and nothing else enforces it. Supervise runs on a real iPhone.

Target check (before every tap)

The model decides from a screen captured seconds earlier; animations, loading or scrolling can move or replace its target in the meantime. Right before tapping, the screen is read again:

  • an element tapped by index is found again by type, label and identifier: tapped where it is now if it moved, not tapped at all (and reported to the model) if it is gone;

  • a point tap aimed at a labeled element is cancelled if a different element is now under that point.

The same check applies to ios_tap in the MCP server.

Traces: ~/.ghosthand/traces/<id>/ holds run.json (task, mode, plan, status, summary, duration, tokens), steps.jsonl (one line per step, with checks) and step-NNN.jpg (the screen the model saw).

From an MCP client: ios_run_task starts the agent in the background, ios_task_status follows it and ios_task_cancel stops it. The API key must then be in the MCP server's environment (claude mcp add … -e ANTHROPIC_API_KEY=…).

Web console

uv run ghosthand ui            # http://localhost:8000, opens the browser
  • Live screen of the simulator or iPhone: click to tap, hold for a long press, drag to swipe; Home, typing, opening an app or a URL; starting and stopping the runner.

  • Agent: run a task (fast or reliable mode, step budget) and follow the plan and steps live.

  • Traces: replay any run step by step (screen the agent saw, reason, outcome, checks, plan); ← → keys to navigate.

  • Elements: the numbered list exactly as the model sees it.

The console listens on 127.0.0.1 only and refuses requests from other hosts or other sites (DNS rebinding, CSRF), since it controls your phone.

API (used by the page and the SDK): GET /api/devices, POST /api/devices/{udid}/runner, GET /api/devices/{udid}/screen.jpg, GET /api/devices/{udid}/elements, POST /api/devices/{udid}/action, GET|POST /api/tasks, GET /api/tasks/{id}, POST /api/tasks/{id}/cancel, GET /api/traces, GET /api/traces/{id}, GET /api/traces/{id}/step-NNN.jpg.

Client SDK (tests, CI)

from ghosthand_client import GhosthandClient

def test_dark_mode():
    result = GhosthandClient().run("Turn on dark mode in Settings", mode="reliable")
    assert result.succeeded, result.summary

See packages/ghosthand-client/README.md.

Tests

uv run pytest        # no device or API key needed: fake model and fake runner

Runner API

Every request must send Authorization: Bearer <token>. runner start creates the token and stores it in ~/.ghosthand/runners/. On the simulator the listener also accepts only connections from this Mac; an iPhone is reached through the CoreDevice tunnel, with the same token.

Route

Purpose

GET /health

status, current app, whether the iPhone is locked

GET /screen

size in points, scale, orientation

GET /screenshot?format=jpeg

full-resolution PNG, or a point-sized JPEG

GET /tree?bundleId=&maxDepth=

raw accessibility tree of the foreground app (+ system alerts)

POST /tap {x, y, duration?}

tap or long press, in points

POST /swipe {fromX, fromY, toX, toY, duration?}

swipe

POST /type {text}

type into the focused field

POST /press {button}

home (volumeUp/volumeDown on an iPhone)

POST /launch {bundleId, relaunch?}

open an app

POST /terminate {bundleId}

quit an app

POST /app-state {bundleIds}

state of each app

POST /known-apps {bundleIds}

installed apps, used to find the foreground one (sent at startup)

POST /open-url {url}

open a URL or deep link

POST /shutdown

stop the runner

Editing the Xcode project

Runner/GhosthandRunner.xcodeproj is generated from Runner/project.yml with XcodeGen: cd Runner && xcodegen generate.

License

MIT, see LICENSE.

Available Tools

14 tools
ios_appsB
Read-only

List installed apps: bundle ID and display name.

NameRequiredDescriptionDefault
deviceNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is covered and the description only needs to add context. It does add the return shape (bundle ID + display name), but says nothing about whether apps come from a booted simulator vs a specific device, or what happens if no device is available.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence, front-loaded with the action and immediately followed by the payload. Nothing is wasted or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list tool with no output schema, the essentials are close to covered, but the undefined device parameter leaves a real gap: an agent cannot tell whether to omit it, pass a UDID, or pass a booted-simulator identifier.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single 'device' parameter has 0% schema description coverage and the description never mentions it — no semantics for what a device value looks like, whether it is required, or what the null default selects. The prose about bundle ID/display name describes output, not the input parameter, so it does not compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List installed apps') and even names the returned fields (bundle ID, display name), which lets an agent distinguish it from ios_devices (devices, not apps) and ios_launch (launching, not listing). It does not explicitly call out those siblings, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the discovery step before ios_launch/ios_terminate, but nothing states when to call it, whether a device must be booted first, or what to do instead for other listing needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_bootA

Get a target ready: boot a simulator (name like "iPhone 17", or UDID) or pick an iPhone, then start the automation runner on it.

The first start compiles the runner (a minute or two). On an iPhone it must be unlocked, and the very first time the user must trust the developer certificate in Settings > General > VPN & Device Management.

NameRequiredDescriptionDefault
deviceYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses the lengthy first compile, the iPhone unlock requirement, and the one-time developer certificate trust step. It does not cover idempotency or what happens if the device is already booted, but the prerequisites are unusually well surfaced.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and follows with relevant prerequisites. The caveats earn their place, and there is little wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter setup tool with no output schema and no annotations, the description covers the key operational context an agent needs. It could mention return value or already-booted behavior, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the single device parameter. It does so by explaining accepted formats: a simulator name like "iPhone 17" or a UDID, plus an iPhone selection option, though the exact accepted value shape remains slightly ambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a concrete verb and resource: boot a simulator or select an iPhone, then start the automation runner. It is distinguishable from siblings like ios_devices and ios_observe, though it does not name them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied as a setup step before automation actions, and the description notes the first-start compile delay. However, it does not explicitly say when to use this versus ios_devices or whether it is required before every session.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_devicesA
Read-only

List simulators and real iPhones, and which ones are ready (booted / reachable).

NameRequiredDescriptionDefault

No parameters

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

readOnlyHint=true already tells the agent this is a safe read, so the description's only added value is naming the readiness states returned (booted / reachable). It does not clarify whether powered-off simulators are included, whether real devices are discovered over the network, or what ordering to expect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with the verb and resource first and the readiness qualifier second. Every clause earns its place; nothing is padded or repeated from the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only list tool with no output schema, stating what is enumerated and what readiness means is essentially sufficient. A sentence on discovery mechanics (e.g., whether it requires Xcode/simctl or scans the network) would close the remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter semantics to document and the baseline of 4 applies. The description correctly introduces no filtering concepts that might imply hidden arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and concrete resources ('simulators and real iPhones'), plus the attribute surfaced (booted / reachable). It is clearly distinguishable from siblings like ios_apps, ios_boot, and ios_observe, which act on a device rather than enumerating devices.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the discovery step before ios_boot/ios_observe, but the description never says 'call this first to obtain a device identifier' or name any alternative or exclusion. No prerequisites or conditions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_launchA

Open an app by bundle ID (see ios_apps). relaunch=True restarts it from scratch.

NameRequiredDescriptionDefault
deviceNo
relaunchNo
bundle_idYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose one real behavioral trait: relaunch=True restarts the app from scratch rather than bringing it to foreground. It is silent on whether the call blocks until the app is running, what happens if the app is not installed, and whether device defaults to a currently booted simulator.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, the core action is front-loaded, and the pointer to ios_apps is inlined rather than padded into a separate sentence. Nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter launch tool with no output schema and no annotations, the description covers the essential action but leaves the device parameter, failure behavior, and post-launch state undocumented. It is adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate: it explains bundle_id (app identifier, discoverable via ios_apps) and relaunch (fresh restart). The third parameter, device, gets no explanation at all, leaving a clear gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Open an app') plus the identifying mechanism ('by bundle ID'), which is enough to separate it from listing tools like ios_apps. It does not explicitly contrast with siblings such as ios_open_url or ios_boot, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical '(see ios_apps)' gives a concrete prerequisite for obtaining the bundle ID, and relaunch is framed as an option. However, nothing says when to prefer this over ios_open_url (deep links) or how it relates to ios_terminate, so usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_observeA
Read-only

See the screen: numbered list of visible elements, plus a screenshot.

Args: screenshot: Include a screenshot (scaled to points, so pixel = tap coordinate). annotate: Draw element indexes on the screenshot. device: Simulator name or UDID; defaults to the only booted one.

NameRequiredDescriptionDefault
deviceNo
annotateNo
screenshotNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnlyHint=true already covering the safety profile, the description adds useful behavioral context: it explains the numbered output, screenshot inclusion, annotation behavior, and the important coordinate mapping that pixel equals tap coordinate. It does not cover failure behavior when no booted device exists or multiple devices are ambiguous, but it is substantially informative.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose, followed by compact parameter notes under "Args:". Every sentence adds operational value with no redundant filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter observation tool with no output schema, the description explains what is returned, how the screenshot relates to tap coordinates, what annotation does, and how device selection defaults. Nothing essential for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry parameter meaning, and it does: screenshot controls screenshot inclusion, annotate controls index drawing, and device accepts a simulator name or UDID with a clear default. The screenshot explanation also adds the critical scaling/coordinate detail not present in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: "See the screen," and immediately specifies the return shape as a numbered element list plus screenshot. This clearly distinguishes observation from action-oriented siblings such as ios_tap, ios_swipe, and ios_type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The observation purpose is implied by "See the screen," so an agent can infer this is the inspection step before interacting. However, the description does not explicitly say when to use it versus alternatives or mention prerequisites such as needing a booted simulator.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_open_urlC

Open a URL or deep link (https://…, maps://…, app schemes).

NameRequiredDescriptionDefault
urlYes
deviceNo

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It doesn't say whether the target device must be booted, which app handles the URL, whether the call blocks until the page loads, or what happens on an unhandled scheme — all material for a device-side navigation action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; the verb, resource, and accepted formats are all in the first clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description should cover preconditions and effects, but it omits device targeting, prerequisite device state, and result behavior. For a two-parameter action tool this leaves significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds useful meaning for `url` via scheme examples, but the `device` parameter — which selects the target simulator/device and defaults to null — is never mentioned or explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (Open) and resource (URL or deep link) and gives concrete scheme examples (https://, maps://, app schemes). This distinguishes it reasonably from ios_launch, though the description never explicitly states the difference between opening a URL and launching an app.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No statement of when to prefer this over ios_launch or how it relates to other device-control siblings. The examples of URL schemes imply usage but never give conditions or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_pressC

Press a hardware button. "home" goes to the home screen.

NameRequiredDescriptionDefault
buttonYes
deviceNo

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses the effect of the home button, but it does not explain device requirements, whether the device must be booted, or any side effects or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with the core action front-loaded. Every sentence contributes directly to understanding what the tool does.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter tool with no annotations and no output schema, the description is still incomplete: it leaves the 'device' parameter undocumented and omits prerequisites such as whether the target device must already be booted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains that button='home' goes to the home screen, but it never mentions the optional 'device' parameter or how target selection works.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Press a hardware button.' It also identifies the one supported button, 'home', and its effect. It is clear against screen taps, but it does not explicitly name sibling alternatives such as ios_tap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus ios_tap or other iOS interaction tools. The only usage context is the implicit fact that this presses a hardware button, with no exclusions or alternative-routing advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_run_taskA

Hand a whole task to the built-in agent (Claude via the Anthropic API) instead of driving step by step.

It runs in the background; poll with ios_task_status. Needs ANTHROPIC_API_KEY (or ant auth login) in the server's environment.

Args: task: What to do, in plain language, e.g. "Turn on dark mode in Settings". max_steps: Step budget before giving up. wait_seconds: Block up to this long for the result (0 returns immediately). mode: "fast" acts directly; "reliable" plans checkpoints and has each one verified (slower, surer). device: Simulator name or UDID.

NameRequiredDescriptionDefault
modeNofast
taskYes
deviceNo
max_stepsNo
wait_secondsNo

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and does well: it discloses background execution, polling with ios_task_status, API-key prerequisites, step budget, blocking wait behavior, and the fast/reliable mode tradeoff. It does not mention device side effects or irreversibility, but the major operational traits are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core distinction, then follows with essential operational notes and a compact Args block. Every section contributes useful information without unnecessary repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no annotations and no output schema, the description covers invocation, prerequisites, background behavior, and all parameters. It leaves some gaps around what device state changes may occur and what the eventual result contains, but the polling reference covers result retrieval adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must document the parameters, and it does: task meaning and example, max_steps as budget, wait_seconds including 0 behavior, mode semantics with both enum values, and device as simulator name or UDID. This adds substantial meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action (hand a whole task to the built-in agent) and resource, and explicitly contrasts it with driving step by step. This lets an agent distinguish ios_run_task from the low-level tap/swipe/type siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage guidance: use this instead of step-by-step control, poll results with ios_task_status, and configure ANTHROPIC_API_KEY or ant auth login. The alternative tool and the background/polling workflow are both named explicitly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_swipeA

Swipe: either a direction from the screen center, or explicit points.

The direction is the finger's: "up" scrolls content down (reveals what is below). To go back in most apps, swipe from x=2 to x=250 at mid-height.

Args: direction: up, down, left or right. from_x: Start x in points (with from_y, to_x, to_y instead of direction). from_y: Start y in points. to_x: End x in points. to_y: End y in points. duration: Gesture duration in seconds; shorter is a faster fling. device: Simulator name or UDID.

NameRequiredDescriptionDefault
to_xNo
to_yNo
deviceNo
from_xNo
from_yNo
durationNo
directionNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so the description carries the full burden — and it discloses two non-obvious traits: direction follows the finger ("up" scrolls content down) and duration controls fling speed. It omits what happens if neither direction nor points are supplied, and says nothing about prerequisites such as a booted simulator or foregrounded app.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the two operating modes before the Args block, and the counterintuitive direction semantics are called out early. The Args block is tight and every line adds meaning; nothing is padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, zero-annotation, no-output-schema gesture tool, the description covers all parameters, both invocation modes, and the main gotcha. It stops short of describing device resolution behavior when device is null and of any failure semantics, which an agent would want before firing a gesture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it mostly does: it enumerates the four direction values, gives units (points) for all four coordinates, flags that point mode is used 'instead of direction', and explains duration semantically. It still doesn't state that the two modes are mutually exclusive by rule, nor mention the 0.3s default or what happens when nothing is passed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Swipe') and immediately splits it into its two modes: a named direction from screen center, or explicit from/to points. That immediately separates it from ios_tap and ios_press, which take single points, though it never names those siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides one concrete recipe ('to go back in most apps, swipe from x=2 to x=250 at mid-height'), which is genuine usage guidance. However, it never states when to choose swipe over ios_tap or ios_press, or when a gesture is inappropriate. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_tapA

Tap an element by [index] from the latest observation, or a point (x, y) in points.

Args: index: Element index from the latest ios_observe or action result. x: Horizontal position in points (with y, instead of index). y: Vertical position in points. hold_seconds: Press duration for a long press (e.g. 1.0); 0 for a normal tap. device: Simulator name or UDID.

NameRequiredDescriptionDefault
xNo
yNo
indexNo
deviceNo
hold_secondsNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full behavioral burden. It does disclose the long-press behavior via hold_seconds and its 0-default, which is genuinely useful. But it says nothing about failure modes (stale index, out-of-bounds coordinates), whether the tap requires a booted device, what the action returns, or whether it blocks until the UI settles.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The intent is front-loaded in the first sentence, with parameters following in a scannable Args block. Every line earns its place; the only mild redundancy is restating the parameter names already in the schema keys.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter UI-mutation tool with no annotations and no output schema, the description covers targeting semantics and long-press duration well enough to invoke correctly. What remains missing is precondition and error behavior, which matters for a device-control action but is a smaller gap than the parameter documentation it does provide.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, yet the Args block documents every one of the five parameters with units ('in points'), the default for hold_seconds ('0 for a normal tap'), the source of index, and the mutual exclusivity of index versus x/y. This fully compensates for the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb (tap) plus two precise targeting modes: an element index from the latest observation, or an (x, y) point in points. An agent can tell this apart from ios_swipe and ios_press by the verb. It stops short of explicitly contrasting itself with the sibling ios_press, but the resource and scope are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implicit workflow guidance is present: the index parameter is described as coming from 'the latest ios_observe or action result', which tells the agent the observation must precede the tap. However, there is no explicit statement of when to choose index vs coordinates, when to prefer ios_press or ios_swipe instead, or what preconditions (booted simulator, fresh observation) must hold.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_task_cancelB

Stop an agent task after its current step.

NameRequiredDescriptionDefault
task_idYes

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses that cancellation happens after the current step, but it omits permissions, reversibility, idempotency, and what happens to task output or state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no wasted words. It immediately states the operation and its key timing constraint.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a task-cancellation mutation with no annotations and no output schema, the definition is too thin. It communicates the core action but leaves important operational details undocumented, especially task_id sourcing and cancellation side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter, task_id, has 0% schema description coverage and is not explained in the description. The phrase 'agent task' loosely connects it to a task identifier, but the description does not state where to obtain it or its expected format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Stop an agent task.' It also clarifies timing with 'after its current step,' which distinguishes this from simply checking status or starting a task, though it does not explicitly name sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this to stop an agent task. However, the description does not say when to use it versus checking ios_task_status, nor does it mention whether already-completed tasks can be canceled.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_task_statusA
Read-only

Status and latest steps of an agent task; without task_id, a list of all tasks.

NameRequiredDescriptionDefault
task_idNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already establishes that this is a safe, non-mutating read. The description adds that the return payload includes 'latest steps' and that omitting task_id returns all tasks, which is useful since there is no output schema, but it says nothing about ordering, volume, or limits on the list-all mode.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single semicolon-joined sentence that front-loads the primary behavior and immediately qualifies the optional parameter. No filler, nothing redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-optional-parameter read tool with a readOnly annotation, the description covers purpose, both operating modes, and the shape of the result despite the absence of an output schema. The remaining gap is provenance of task_id and any bound on the listing mode.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and task_id has no description in the schema, so the description must carry the meaning — and it does, by explaining the omitted/null default behavior (list all tasks). What it omits is where a task_id comes from or its format, preventing a top score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific resource (an agent task) and states exactly what is returned: status plus latest steps. It also covers the dual retrieval mode, so an agent can tell it apart from the mutating siblings ios_run_task and ios_task_cancel, though it does not name them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'without task_id, a list of all tasks' gives an explicit condition for choosing between the single-task and list modes, which is real usage guidance. However, it never states when to prefer this over ios_run_task (to check progress after starting) or ios_task_cancel, leaving sibling routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_terminateC

Quit an app.

NameRequiredDescriptionDefault
deviceNo
bundle_idYes

TDQS

C2.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full behavioral burden. It doesn't say whether the app's unsaved state is discarded, whether the kill is forced or graceful, what happens if the app isn't running, or what errors can occur -- all critical for a destructive mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three words is not conciseness here, it is under-specification; nothing is front-loaded because there is essentially no content. Every needed detail was omitted rather than compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With two undocumented parameters, no annotations, and no output schema, the description needed to carry everything and instead says almost nothing. An agent cannot reliably call this tool from the definition alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description mentions neither parameter. It doesn't clarify that bundle_id must match an installed app (obtainable via ios_apps) or what the optional device selector means in a multi-device context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Quit an app" gives a verb plus resource, so the basic intent is inferable, but it never says this is the inverse of ios_launch or that the target is an installed iOS app identified by bundle id. Compared with siblings like ios_launch and ios_apps, the distinction is only implied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this versus ios_launch, ios_apps, or the task tools, and no preconditions (e.g., app must be running, device targeting). An agent must guess at the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ios_typeA

Type text into the field that has keyboard focus (tap the field first). Use "\n" for Return.

NameRequiredDescriptionDefault
textYes
deviceNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It usefully discloses the focus prerequisite and the "\n" newline convention, but is silent on whether typed text appends or replaces existing content, what error occurs without focus, and what the call returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action and target, with the prerequisite and newline convention packed into compact parentheticals. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter typing tool with no output schema, the core interaction is adequately described. The unexplained `device` parameter and the absence of any failure/return behavior leave meaningful gaps given zero annotation coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain both parameters. It partially covers `text` via the newline note but says nothing about `device` — an agent cannot tell whether it selects a simulator, a physical device, or defaults to a single connected target.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (type) and target (the field with keyboard focus), which cleanly separates it from ios_tap, the tool the parenthetical tells you to call first. The distinction from siblings is implied rather than named outright, which keeps it just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear precondition — the field must already have keyboard focus, so tap it first — plus the convention for sending Return. It does not address failure cases (e.g., what happens with no focus) or when to prefer ios_press or ios_open_url instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 14 tool updatesv0.1.0
    • First observedios_apps
    • First observedios_boot
    • First observedios_devices
    • First observedios_launch
    • First observedios_observe
    • First observedios_open_url
    • First observedios_press
    • First observedios_run_task
    • First observedios_swipe
    • First observedios_tap
    • First observedios_task_cancel
    • First observedios_task_status
    • First observedios_terminate
    • First observedios_type

TDQS

B3.4/5.0

Scored across 14 tools

Disambiguation4/5

Each tool maps to a distinct action in the observation→interaction loop (observe, tap, swipe, type, press), and the agent-mode tools (ios_run_task / ios_task_status / ios_task_cancel) are clearly separated from manual driving. The only soft overlaps are ios_launch vs ios_open_url (an app scheme URL could effectively launch an app) and ios_press vs dedicated navigation, which an agent could occasionally misselect.

Naming Consistency4/5

All 14 tools use a uniform ios_ prefix with snake_case, giving a highly predictable namespace. Most are verb or verb_noun (ios_boot, ios_tap, ios_open_url, ios_run_task), with a few noun-only list/query names (ios_devices, ios_apps, ios_task_status) that follow a conventional read-only pattern but are a minor deviation.

Tool Count5/5

14 tools sits squarely in the well-scoped range and every tool maps to a needed primitive: discovery, boot, observation, four input gestures, app lifecycle, URL handling, plus an optional agent layer with status/cancel. Nothing looks redundant or padded for the scope of iOS device automation.

Completeness4/5

The core loop is fully covered end-to-end: list devices, boot, observe screen (elements + screenshot), interact (tap/swipe/type/press), and app lifecycle (apps, launch, terminate, open_url), plus an agent mode with status and cancellation. Minor gaps remain—no install/uninstall of apps, no orientation/rotation, no explicit wait/sleep or clipboard/keyboard-dismiss primitive—but agents can work around these with existing gestures.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers