phone-agent-mcp

Let an AI agent (Claude Code, or any MCP client) see and operate a real phone from your Mac: iPhones and iPads over USB or Wi-Fi, Android phones and Android TVs over adb. Nothing is installed on the phone. On iOS it uses the same device services Xcode uses for screen mirroring (CoreDevice), through pymobiledevice3.

It is a local MCP server (stdio), with an optional shared daemon and a small web console: a live view of the phone, the action log, and a kill switch.

Unofficial. This relies on undocumented Apple services that can change in any iOS or Xcode update. It is not affiliated with or endorsed by Apple, Google or WhatsApp. Use it only on devices you own. Automating a messaging app may be against that app's terms of service: that is your responsibility. Read the safety notes below before letting an agent near your phone.

What it can do

Android TV has no touch: the agent navigates with press_key (up, down, left, right, select, back, home, play_pause).

Related MCP server: ghost-in-the-droid

Requirements

• A Mac with macOS 26+ and Xcode 27+, opened once with the phone connected (it installs the developer tools on the phone).

• iPhone/iPad on iOS 27+, trusted by the Mac, with Developer Mode on (Settings > Privacy & Security).

• Python 3.11+, and swiftc (comes with Xcode) to build the screen-text helper.

• Optional: adb for Android (brew install android-platform-tools).

Install

git clone https://github.com/uD000/phone-agent-mcp.git
cd phone-agent-mcp
./setup.sh                       # venv, dependencies, helpers, then a health check

./setup.sh ends by running venv/bin/phone-agent doctor, which tells you exactly what is missing (Xcode version, Developer Mode, the phone's screen and touch services, helpers). Then register the server with Claude Code and start a new session:

claude mcp add phone-agent -- "$PWD/venv/bin/python" -m phoneagent.mcp_server

Ask it to "list my phones", then "take a screenshot".

Check it works

venv/bin/python tests/test_offline.py        # no phone needed
venv/bin/python tests/test_protocol.py       # starts the real MCP server, no phone needed
venv/bin/python scripts/smoke_live.py        # your phone: connect, screenshot, OCR, Home. Sends nothing.

Live view and controls

The MCP server hosts a console on 127.0.0.1:47831. Open it with a single-use login link (your token never goes in a URL):

venv/bin/phone-agent console

It shows the live screen (up to the ~55 fps the phone's video offers), the action log, the policy, a Wi-Fi quality selector and the kill switch. For several agents sharing one phone session, run venv/bin/phone-agent serve and point clients at the daemon.

Safety model

All of this is controlled by you, not the agent:

• Approvals. Anything that sends, posts, buys or deletes needs your click in a native macOS dialog. The agent cannot click it and its own approved=true is ignored. If you deny or do not answer, nothing happens.

• Credentials. Passwords, codes and card numbers are entered by you in a private dialog, typed into the phone by the server, and never appear in tool results, screenshots (withheld if a secret is readable on screen) or the audit log.

• Kill switch. bin/pa-kill (or the console button) stops all input at once. Turning it off needs your confirmation.

• Passcode / Face ID prompts are human-only; taps and typing are refused while one is on screen.

• Policy in ~/.phone-agent/policy.json (phone-agent policy show|set): approval mode, app allow/deny lists, rate limit.

• Audit log in ~/.phone-agent/audit.jsonl: every tool call; never secrets, clipboard text or traffic results.

• The console/daemon listens on loopback only, checks the Host header, and needs a bearer token or a single-use code.

Honest limits. These guards stop what an agent can do through the tools. An agent that also has unrestricted shell access on the same Mac could edit ~/.phone-agent/policy.json or delete the kill-switch file: run untrusted agents in a sandbox. The app allow-list only governs launching apps, not where taps land. Packet capture needs a USB cable (iOS refuses it over Wi-Fi). The screen stream can freeze on Wi-Fi; the session detects that and rebuilds it. See DESIGN.md for the full design.

Layout

src/phoneagent/: session.py (tunnel, stream, input) · mcp_server.py (tools) · console_app.py, daemon.py · policy.py, approval.py, killswitch.py, secrets.py · services/ (layout, apps, clipboard, observe) · skills/ · vision.py (OCR) · android.py. tools/ holds the Swift helpers (built into bin/ by setup.sh).

License

GPL-3.0-or-later (see LICENSE), because it links against pymobiledevice3 (GPL-3.0). Portions are adapted from open-iphone-use (Apache-2.0); see NOTICE and licenses/.