# iphone-mcp

Control a **real, physical iPhone** from an MCP client (Claude Code, Claude Desktop, or any
MCP host). Not a simulator, not iPhone Mirroring — the actual device, driven through
Apple's own UI-automation channel.

```
Read your messages · reply to people · reorganise your home screen · open apps
navigate any app · fill in forms · scroll, tap, swipe, drag
```

---

## Why this works at all

iOS gives exactly one sanctioned way to inject touch events into a physical device:
**XCTest / XCUITest**, running as a UI-test bundle on the phone. That is what
[WebDriverAgent](https://github.com/appium/WebDriverAgent) is — an XCTest runner that
exposes an HTTP server on the device.

Everything else in the iOS tooling world (`devicectl`, `pymobiledevice3`,
`libimobiledevice`, `idb`) does device *management* — install apps, launch them, take
screenshots, read logs — but **cannot tap**.

```
MCP client
    │  stdio (MCP)
    ▼
iphone-mcp  ─────────────────┬──────────────────────────┐
    │  HTTP :8100            │ devicectl                │ sqlite (read-only)
    ▼                        ▼                          ▼
iproxy (usbmux over USB)   app inventory        ~/Library/Messages/chat.db
    │
    ▼
WebDriverAgentRunner (XCTest) on the iPhone
    │
    ▼
iOS · SpringBoard · any app
```

## The design decision that makes it usable

The obvious approach — screenshot the phone and let the model guess pixel coordinates — is
where phone agents fail in practice. This server does what works for browser automation
instead: it turns the accessibility tree into a **compact outline with stable refs**, and
the model acts on refs.

The numbers make the case. One iPhone home screen serialises to:

| | raw XCUIElement JSON | this server's snapshot |
|---|---|---|
| size | **764 KB** | ~1.2 KB |
| nodes | 524 | 21 |
| tokens | ~190,000 | ~400 |

and the output is directly actionable:

```
app: com.apple.MobileSMS | screen: 440x956pt | snapshot #3 | 72 elements

[e1] NavigationBar "Messages" @220,90
  [e2] Button "Edit" @52,84
[e9] Cell "Keka, Unread, i don't have a good headshot for my linkedin, Pinned" @220,185
[e12] TextField "iMessage" value="" @220,905
[e13] Button "Send" @410,905
```

The model calls `ui_tap({ref: "e13"})`. It never sees a coordinate. Screenshots stay
available via `ui_screenshot` for when the tree isn't enough (canvas UI, games, visual
confirmation).

---

## Requirements

- **macOS** with **Xcode 15+** (26 recommended) and the command line tools
- An **iPhone** running iOS 17+ (developed and tested against **iOS 26.5**)
- An **Apple Developer account** — a free personal team works, but certificates expire
  after 7 days and WDA must be rebuilt. A paid account with a wildcard profile is far less
  friction.
- `brew install libimobiledevice` (provides `iproxy` and `idevice_id`)
- **Node 22+** (uses the built-in `node:sqlite`)

On the phone: **Settings → Privacy & Security → Developer Mode → on**, then trust this Mac
when prompted.

## Setup

```bash
git clone https://github.com/teddyoweh/iphone-mcp.git
cd iphone-mcp
npm install
npm run build

bash scripts/fetch-wda.sh     # clone WebDriverAgent into vendor/
bash scripts/wda-up.sh        # build, sign, install, launch, wait for ready
```

`wda-up.sh` is idempotent — it exits immediately if WDA is already answering on :8100.
Re-run it after a reboot, a cable reconnect, or if iOS jetsams the runner.

It auto-detects your Apple Developer Team ID from your installed provisioning profiles.
Override it if that guesses wrong:

```bash
export IPHONE_MCP_TEAM_ID=XXXXXXXXXX
```

### Register with your MCP client

Claude Code:

```bash
claude mcp add iphone -- node /absolute/path/to/iphone-mcp/dist/index.js
```

Or in `claude_desktop_config.json` / `.mcp.json`:

```json
{
  "mcpServers": {
    "iphone": {
      "command": "node",
      "args": ["/absolute/path/to/iphone-mcp/dist/index.js"],
      "env": {
        "IPHONE_MCP_PASSCODE": "123456"
      }
    }
  }
}
```

Then ask it to `iphone_status` — that health-checks the whole chain.

### Environment variables

| variable | default | meaning |
|---|---|---|
| `IPHONE_MCP_PASSCODE` | — | Device passcode, typed on the on-screen keypad to unlock. Prefer this over passing it in a tool call so it never lands in conversation logs. |
| `IPHONE_MCP_UDID` | auto | Device UDID, if more than one is attached. |
| `IPHONE_MCP_WDA_URL` | `http://127.0.0.1:8100` | Where WebDriverAgent is reachable. |
| `IPHONE_MCP_TEAM_ID` | auto | Apple Developer Team ID for signing WDA. |
| `IPHONE_MCP_AUTOSTART` | `1` | Set `0` to stop the server shelling out to `wda-up.sh` when WDA is unreachable. |

### Permissions

- **Full Disk Access** for whatever process runs the server (Terminal, Claude, …) — needed
  to read `chat.db` and Contacts. System Settings → Privacy & Security → Full Disk Access,
  then restart the app.
- Messages tools degrade gracefully without it; everything else still works.

---

## Tools

**Observing**

| tool | what it does |
|---|---|
| `iphone_status` | Health check: WDA, iOS version, screen, lock state, foreground app, Messages DB. |
| `ui_snapshot` | The primary tool. Compact ref-addressed outline of the screen. Supports `query` filtering. |
| `ui_screenshot` | PNG of the screen, downscaled. For when the tree isn't enough. |

**Acting**

| tool | what it does |
|---|---|
| `ui_tap` | Tap an element by ref. Optional double-tap. |
| `ui_tap_point` | Escape hatch: tap absolute coordinates **in points**. |
| `ui_type` | Type into a field (optionally tapping it first), with `clear_first` / `submit`. |
| `ui_swipe` | Swipe/scroll in a direction, optionally scoped to a scrollable element. |
| `ui_long_press` | Press and hold — opens context menus, enters home screen edit mode. |
| `ui_drag` | Press, hold, drag one element onto another. |

**System**

| tool | what it does |
|---|---|
| `app_open` / `app_close` / `app_list` | Launch, terminate, enumerate apps by name or bundle id. |
| `open_url` | Any URL scheme. Often the most reliable navigation: `sms:`, `tel:`, `prefs:root=WIFI`. |
| `press_button` | `home`, `volumeup`, `volumedown`. |
| `device_lock` | Lock, or wake and unlock (types the passcode on the keypad). |

**Messages**

| tool | what it does |
|---|---|
| `messages_threads` | Recent conversations with names, previews, unread counts. |
| `messages_read` | Full history of one conversation, by contact name / number / email. |
| `messages_draft` | Compose a reply and hold it. **Sends nothing.** |
| `messages_send` | Actually send a draft. Requires `confirm_text` to match exactly. |

**Home screen**

| tool | what it does |
|---|---|
| `home_map` | Walk every page; report apps, folders, widgets, badges and the dock. |
| `home_remove_app` | Remove an icon from the home screen. App stays installed. *Experimental in bulk — see Status.* |
| `home_move_app` | Drag an icon to a page/row/column. Best effort, verified. *Experimental.* |
| `home_page_editor` | Open the page editor, where whole pages can be hidden. |

---

## Notes from building this

Things that cost real debugging time, documented so they don't cost yours.

**Points vs pixels.** Every coordinate WDA accepts or returns is in *points*
(440 × 956 on a 17 Pro Max). Screenshots come back in *pixels* (1320 × 2868, 3×). Mixing
them is a silent 3× error. Only `screenshot()` is allowed to know pixels exist.

**85% of your messages aren't in `message.text`.** On current macOS that column is NULL for
most rows and the content lives in `attributedBody`, an NSArchiver *typedstream* blob. On
the machine this was built on: 193,479 of 226,918 rows. A reader that only checks `.text`
silently loses most of the history. The decoder in `src/messages.ts` was validated at
**400/400 exact matches** against rows that happen to have both fields populated.

**`message.date` overflows a JS number.** It's nanoseconds since 2001 — e.g.
`806973797577999872`, well past `Number.MAX_SAFE_INTEGER`. `node:sqlite` throws rather than
silently rounding, so those queries must opt into BigInt.

**Auto-lock is the number one cause of mysterious failures.** When the phone locks itself,
iOS revokes the XCTest process's UI-testing authorization, and WDA then fails every call
with errors that never mention locking:

```
Failed to get system application: Not authorized for performing UI testing actions
stale element reference: ... Application 'local.pid.0' is not present
Failed to resolve query: Application local.pid.0 is not running
```

A batch that starts fine and then fails on every subsequent step has almost certainly just
hit the auto-lock timer — chasing the "stale element" message instead leads nowhere. Every
tool here detects this class of error, unlocks, and retries once, which is why
`IPHONE_MCP_PASSCODE` matters for anything long-running.

**`pressButton("home")` does not wake a Face ID phone.** The display stays asleep and every
screenshot comes back solid black. `/wda/unlock` does wake it.

**The lock screen tells you the passcode length.** The passcode field's accessibility value
reads `"4 of 6 values entered"` — so a wrong-length passcode can be rejected *before*
submitting a guaranteed-failed attempt, and stale digits can be cleared first.

**Home screen pages render lazily.** Off-screen page containers exist in the tree, but
SpringBoard doesn't populate their icons, so mapping requires actually swiping. There's no
shortcut.

**Double-tap needs a gap.** Two taps with no delay aren't read as a double-tap; ~130 ms
between them is.

---

## Safety

- **Sending messages is gated.** `messages_draft` has no side effect; `messages_send` is a
  separate tool that requires the draft id *and* an exact copy of the text. The intended
  flow is: draft → show the user → send only on approval.
- **`home_remove_app` never deletes an app.** iOS puts "Delete App" (destroys data)
  directly next to "Remove from Home Screen" (keeps it in App Library) in the same menu.
  The matcher for that sheet is exact and anchored; if it can't find an unambiguous
  "Remove from Home Screen", it taps Cancel and reports rather than guessing.
- **Your passcode belongs in the env**, not in tool arguments.
- This drives a real phone that sends real messages to real people. Treat it that way.

## Status

Verified working against a physical iPhone 17 Pro Max on iOS 26.5:

- `ui_snapshot`, `ui_screenshot`, and the whole ref-addressed tap/type/swipe loop
- `app_open` / `app_list` / `open_url` / `press_button`
- passcode unlock, including rejecting a wrong-length passcode before submitting it
- `messages_threads` / `messages_read` — contact names, group participants, unread counts,
  `attributedBody` decoding
- `home_map` across all pages

**Experimental — do not trust in bulk:** the home screen *mutation* tools
(`home_remove_app`, `home_move_app`, `home_page_editor`). A single removal works, but
running many in sequence has been observed to destabilise WebDriverAgent itself until the
runner is relaunched. Go a few at a time and verify with `home_map`.

## Limitations

- USB only in this configuration. WDA over Wi-Fi works but is slower and flakier.
- Face ID cannot be triggered programmatically — passcode entry is the only unlock path.
- Cross-page icon drags are unreliable. `home_move_app` verifies and tells you the truth
  rather than claiming success.
- Removing icons doesn't reflow icons backward across pages — iOS never does that — so
  decluttering reduces icons per page more than it reduces page count. Use
  `home_page_editor` to hide pages outright.
- `home_map` takes ~4 s per page. Some MCP clients default to a 60 s tool timeout; raise it
  or pass a smaller `max_pages`.
- Free Apple developer accounts need WDA rebuilt weekly.

## Development

```bash
npm run build     # compile
npm run watch     # compile on change
npm run wda       # bring WebDriverAgent up
```

`vendor/WebDriverAgent`, `.wda-build/`, and `dist/` are gitignored.

## License

MIT