Universal Phone Harness

Universal Phone Harness is a Python MCP server, CLI, and library for controlling Android and iOS devices from AI agents. It turns accessibility hierarchies into bounded indexed DOMs, grounds actions to observed generations, verifies postconditions, and blocks unconfirmed destructive targets.

Android uses ADB and UIAutomator. The iOS backend uses WebDriverAgent. A deterministic mock backend supports local development and CI.

Features

• 18 MCP tools, native MCP Resources, and MCP Prompts for observation, taps, typing, gestures, navigation, waits, assertions, dialogs, clipboard access, screenshot saving, and health checks

• Indexed UI elements with stale-observation rejection

• Accessibility-first grounding with local OCR fallback for canvas and WebView surfaces

• Explicit post-action assertions and one re-grounded retry

• Destructive-action gates, step budgets, loop detection, and PII masking

• Hard output budgets: 600 estimated tokens for observations and 200 for post-action DOMs by default

• Compact JSON and native MCP image content

• Android, WebDriverAgent-based iOS, and deterministic mock backends

• CLI and Python APIs for direct automation

Related MCP server: Mobile Device MCP

Measured results

Tests on a Huawei JLN-LX1 running Android 12 used one warm-up and five measured samples:

The clear baseline reproduces the earlier incomplete delete sequence. The comparison measures the cost and speed of the corrected implementation, not an overall system speedup. Full samples and assertions live in artifacts/android-hardening-benchmark.json.

Live MCP measurement reduced one Settings post-action DOM from 2,313 to 751 characters, a 67.53% reduction. Image requests now use MCP image blocks instead of embedding base64 inside JSON text.

Requirements

• Python 3.10 or newer

• Android: USB debugging or wireless ADB authorization

• iOS: a reachable WebDriverAgent server

• Optional OCR: rapidocr-onnxruntime and OpenCV

Installation

git clone https://github.com/ZachDreamZ/universal-phone-harness.git
cd universal-phone-harness
python -m pip install -e ".[dev]"

Install OCR support when needed:

python -m pip install -e ".[ocr]"

Android setup

1. Enable Developer options and USB debugging.

2. Connect and authorize the phone.

3. Confirm ADB sees it:

adb devices -l
phone-harness doctor

The default backend is Android and fails closed. It will not silently substitute the mock backend when no device is connected.

MCP setup

After installation, register this stdio server in any MCP client:

{
  "mcpServers": {
    "phone-harness": {
      "command": "python",
      "args": ["-m", "phone_harness.mcp_server"]
    }
  }
}

AI Agent Quick-Install Prompt

To install and wire this harness into any AI agent (Claude Code, Antigravity, Cursor, Windsurf, OpenCode, Cline), copy and paste this prompt:

"Configure and install universal-phone-harness as an MCP server:

1. Clone https://github.com/ZachDreamZ/universal-phone-harness.git and install editable with pip install -e ..

2. Register the MCP server in my agent's MCP configuration (e.g., claude_desktop_config.json, .gemini/antigravity-cli/settings.json, or .cursor/mcp.json):

   {
     \"mcpServers\": {
       \"phone-harness\": {
         \"command\": \"python\",
         \"args\": [\"-m\", \"phone_harness.mcp_server\"]
       }
     }
   }

   (Note: Add \"--device-type\", \"mock\" to args for deterministic offline testing without physical hardware).

3. Verify server connectivity via the phone://device/status resource or by running phone-harness doctor."

Call phone_observe before indexed actions. Pass its observation_generation with the index:

{
  "index": 3,
  "observation_generation": 12
}

Text and coordinate selectors do not require a generation. Typed text is not echoed in MCP responses.

Remote MCP Server (SSE & HTTP)

Run the harness as a remote HTTP/SSE server for cloud AI agents (Modal, Fly.io, AWS, LangChain) with Bearer token security:

phone-harness serve --transport sse --host 0.0.0.0 --port 8080 --auth-token YOUR_SECRET_TOKEN

Connect MCP clients to http://<HOST>:8080/sse with Authorization: Bearer YOUR_SECRET_TOKEN.

CLI

phone-harness doctor
phone-harness observe
phone-harness tap Settings
phone-harness settings wifi
phone-harness swipe up --distance medium
phone-harness press HOME
phone-harness wait --text Connected --timeout 5000
phone-harness screenshot screen.png --som
phone-harness clipboard --set "My OTP"
phone-harness report --optimal-steps 5

# Remote MCP server (HTTP/SSE)
phone-harness serve --transport sse --port 8080 --auth-token secret-token

# Deterministic trace replay with layout self-healing
phone-harness replay session.trace.jsonl --self-heal

# Autonomous app crawler & QA auditor
phone-harness crawl --package com.example.app --max-depth 5 --budget 25 --output-dir ./audit

# Perceptual settle detection
phone-harness settle --timeout 2.0

Python API

from phone_harness import ActionRequest, ActionType, PhoneHarness, VerificationSpec

harness = PhoneHarness()
state = harness.observe()

settings = next(element for element in state.elements if element.text == "Settings")
result = harness.execute_action(
    ActionRequest(
        action=ActionType.TAP,
        target_index=settings.id,
        observation_generation=state.generation,
        verify=VerificationSpec(assert_app_package="com.android.settings"),
    )
)

print(result.new_state.current_app_package)

Use the mock backend explicitly in tests:

from phone_harness.backends.mock import MockPhoneDevice
from phone_harness.harness import PhoneHarness

harness = PhoneHarness(device=MockPhoneDevice())

Other explicit backend choices:

from phone_harness.core.config import HarnessConfig

android_config = HarnessConfig(default_platform="android")
ios_config = HarnessConfig(default_platform="ios")
mock_config = HarnessConfig(default_platform="mock")

allow_mock_fallback=True is opt-in because silent fallback can make an agent believe it controls a real device when it does not.

MCP tools

MCP Resources

Expose real-time device state as native MCP resources without consuming action step budgets:

MCP Prompts

Built-in agent workflows accessible via MCP prompt templates:

Architecture

phone_harness/
  backends/        Android, iOS, and mock device adapters
  core/            Models, configuration, device interface, exceptions
  engine/          Verification, safety, dialogs, sessions, trajectories
  perception/      Tree compaction, OCR matching, visual detection, SoM
  plugins/         Example MCP client configurations
  cli.py           Command-line interface
  harness.py       Action and observation orchestration
  mcp_server.py    MCP JSON-RPC stdio server
tests/             Unit and integration tests
benchmarks/        Reproducible connected-device benchmark

Development

python -m pip install -e ".[dev]"
python -m pytest -q
python -m compileall -q phone_harness benchmarks
python -m build
python -m twine check dist/*

The test suite contains 108 tests and does not require connected hardware. verify_all.py adds optional live-device checks.

Security

Report vulnerabilities privately. See SECURITY.md. Do not include credentials, phone contents, screenshots, or private device identifiers in reports.

Contributing

See CONTRIBUTING.md. Contributions require tests and must pass CI.

License

MIT. See LICENSE.