Skip to main content
Glama


The pitch, in one breath

Every AI agent can think. Almost none can touch a phone.

Ghost is the body. It gives any LLM agent a real Android phone or iPhone: it reads the screen, taps, swipes, and types through a clean tool surface, and it scales from one device on your desk to a whole phone farm. Point Claude Code, Codex, Antigravity, Cursor, your LangChain app, or a model running inside the phone itself at it, and your agent grows hands.

Bring your own brain. Keep the same body. MIT, forever.


Related MCP server: adb-mcp

Mix and match: the Ghost matrix

Ghost 1.3 is the only Android + iOS agent framework where platform, brain, and driver are all swappable. Pick one from each column. They all compose.

Pick a platform

Pick a brain

Pick a driver

Android over USB ADB

Claude Code (free with your Max/Pro sub)

MCP client (Claude Code, Codex, Antigravity, Cursor, Opencode)

Android over Wi-Fi ADB

Anthropic API

LangChain toolkit

Android in a Docker + KVM emulator pool

OpenRouter (any model, one key)

LlamaIndex tool spec

iPhone over Appium + WebDriverAgent

Ollama (fully local)

Ghost CLI (ghost "book a table" --device pixel)

iPhone over Tailscale + WebDriverAgent (wireless)

vLLM (your own GPU)

Web dashboard chat with a live phone stream

On-device — llama.cpp (.gguf, any quant), MediaPipe (.task, Gemma mobile bundles), MLX (Apple Silicon, opt-in). The model runs in the phone, airplane-mode works

REST API (/docs OpenAPI)

The Docker + KVM emulator pool is Android-only; iOS uses Apple's own simulator and Xcode signing, which are macOS-native.

The body stays the same 66 MCP tools no matter what you plug in. Every new model release is a free upgrade to your phone agent.


See it in action

Nine demos, one per feature, each recorded on real devices. Click any clip for the full mp4.


What It Does

Open-source Python framework for controlling Android and iOS devices from one agent harness. Android runs through ADB. iOS runs through Appium XCUITest and WebDriverAgent, with real iPhones as the target path and simulators for development and CI.

Define skills for any app, run them from the dashboard or API, scale across a phone farm.

The ghost taps what you'd tap

  • Android control through ADB: tap, swipe, type, clipboard, shell, intents, and stealth variants

  • iOS control through WebDriverAgent: screenshot, accessibility tree, tap, swipe, type, app launch, clipboard, browser actions

  • Live phone screen streaming: Android MJPEG/WebRTC, iOS WDA MJPEG with screenshot fallback

  • Interactive touch-to-tap on the streamed screen

  • Multi-device phone farm with per-device job queues

Forge reusable skills for any app

  • YAML-based UI element definitions per app

  • Platform-specific selectors with elements.yaml for Android and elements_ios.yaml for iOS

  • Python action classes with precondition checks

  • Multi-step workflows that chain actions together

  • Built-in skills for TikTok and Play Store

  • iOS browser/news demo skill and smoke-level TikTok iOS workflows

  • Skill Hub: browse, search, and install skills from the community registry

  • Install from CLI: android-agent skill install tiktok

Teach the ghost new tricks

  • BFS-based auto app explorer: discovers every screen and transition

  • LLM-assisted Skill Creator: chat with AI while viewing the live device stream

  • The AI identifies UI elements and generates action/workflow code

Scale the haunting

  • Multi-device phone farm with per-device job queues

  • Bot runner: queue, schedule, and monitor automation jobs

  • Per-device integration tests with Android screen recording or iOS WDA MJPEG recording

Platform Support

Surface

Android

iOS

Device ref

ADB serial, e.g. emulator-5554

ios:<udid>

Backend

ADB + optional Portal companion app

Appium XCUITest + WebDriverAgent

Real device support

Yes

Yes, with Mac/Xcode/WDA signing

Simulator/emulator support

Android emulator tooling

Booted iOS simulators via Appium/WDA

Live stream

Portal WebRTC, MJPEG, screencap

WDA MJPEG, screenshot polling fallback

Screen tree

Android UIAutomator XML

Normalized XCTest accessibility tree

Skills

elements.yaml, Android packages

elements_ios.yaml, iOS bundle IDs

Android-only today

ADB shell, intents, wireless ADB, Play Store helpers, Portal overlay

Unsupported with stable platform errors


Why it is built this way

Skills cost $0. Thinking costs tokens. An agent should not pay an LLM to tap a button it has tapped a thousand times. Known workflows compile to skills: deterministic YAML + Python recipes that replay in seconds with zero LLM calls. Use AI for the unknown task, replay for the known one.

The phone is the easiest sandbox there is. Give the ghost an old Android with its own SIM and its own accounts, physically separate from your life. You already know how to set up a phone.

Zero-app by default. A fresh install touches nothing on the Android device: screen reads go through uiautomator, actions through adb shell input, no root, no accessibility service. Want it faster? The optional Portal companion app gives roughly 30x quicker UI reads. Want it private? On-device mode runs the whole loop inside the phone, nothing leaves it. All opt-in.

Local-first, cloud-optional. The server and dashboard run on your machine. With a local or on-device brain, your screenshots never leave your network. Pick a cloud brain and prompts go to that provider, same as any tool. Your call, always.


Requirements

  • Python 3.10+

  • Android: Android phone with USB debugging enabled and ADB on PATH (adb devices should list your phone)

  • iOS: macOS with Xcode, Appium 2 + XCUITest driver, and a trusted iPhone or booted simulator

  • Node.js 18+ (for the frontend dev server)


iOS Support

Ghost drives iPhones through Appium/WebDriverAgent with the same tool surface: ios:<udid> device refs route tap/swipe/type/screenshot to WDA, and iOS-aware browser primitives (open_url, read_news, extract_visible_text) cover web tasks. Android-only tools (shell, launch_intent, Portal overlay) return a clear platform error instead of failing silently.

iOS is opt-in: enable it with GITD_ENABLE_IOS=1 (or ios_platform_enabled=true in .env). Wireless drive rides your Tailscale tailnet, so treat the tailnet as the trust boundary. See docs/SETUP_IOS.md for Appium/WDA setup, and on-device on iPhone to run the model on the iPhone itself.


Quick Start

Zero-install with uvx (or pipx):

uvx ghost-in-the-droid doctor   # green/red preflight: python, adb on PATH, devices, ports, LLM keys
uvx ghost-in-the-droid login    # sign in with your Claude subscription, no API key needed
uvx ghost-in-the-droid up       # server + dashboard at http://localhost:5055  (API docs at /docs)

doctor prints a checklist with fix hints instead of a stack trace when something is missing (e.g. adb not on PATH). Prefer pipx? pipx install ghost-in-the-droid gives you the ghost-in-the-droid / android-agent commands.

Sign in with your Claude subscription (no API key)

If you have a Claude Max/Pro subscription, you don't need an API key. android-agent login signs you in through the claude CLI's own Anthropic OAuth flow and points Ghost at the claude-code provider:

android-agent login       # opens Anthropic sign-in via the claude CLI

Ghost never handles or stores your token; the claude CLI owns it, refresh included. doctor shows a green Claude subscription check once you are signed in. To use an API key instead, set ANTHROPIC_API_KEY (or OPENAI_API_KEY / OPENROUTER_API_KEY) and pick that provider.

Select Ollama in the Phone Agent tab. Runs entirely on your machine with Ollama:

brew install ollama       # or: curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull llama3.2:3b   # 2GB, fast, good tool-use
git clone https://github.com/ghost-in-the-droid/android-agent.git
cd android-agent
pip install -e ".[all]"

android-agent doctor        # preflight
android-agent up            # start server + dashboard on :5055

# Frontend (separate terminal)
cd frontend && npm install && npx vite --host 0.0.0.0 --port 6175
# Dashboard at http://localhost:6175

iOS Quick Start

iOS support requires a Mac because Appium uses Xcode's XCUITest/WebDriverAgent stack. Real iPhones also require trust, Developer Mode, UI Automation permission when prompted, and WDA signing with an Apple development team.

# 1. Install and run Appium XCUITest
npm install -g appium
appium driver install xcuitest
appium --base-path /

# 2. Find your iPhone or booted simulator UDID
xcrun xctrace list devices
xcrun simctl list devices booted

# 3. Configure the backend for iOS
export IOS_DEVICE_UDID="<udid>"
export IOS_APPIUM_URL="http://127.0.0.1:4723"
export IOS_BUNDLE_ID="com.google.chrome.ios"       # or com.apple.mobilesafari
export IOS_MJPEG_SERVER_PORT="9100"                # use unique ports per iOS device

# 4. Run a product-path smoke workflow
uv run python scripts/ios_chrome_news_smoke.py \
  --device "ios:<udid>" \
  --bundle-id "$IOS_BUNDLE_ID" \
  --url https://text.npr.org/ \
  --max-headlines 5 \
  --max-articles 3 \
  --fix-health \
  --out-dir data/ios_chrome_news_smoke

For full real-device signing, simulator, WDA MJPEG, health recovery, scheduler, and MCP setup details, see docs/SETUP_IOS.md.

Environment Variables

Copy .env.example to .env (if provided) or create a .env file in the project root. The server reads configuration via Pydantic Settings. Optional variables include:

Variable

Purpose

OPENAI_API_KEY

LLM features (Skill Creator, Agent Chat)

ANTHROPIC_API_KEY

Alternative LLM provider

OPENROUTER_API_KEY

OpenRouter LLM provider

DEFAULT_DEVICE

ADB serial (auto-detected if empty)

IOS_DEVICE_UDID

iPhone or simulator UDID; devices are addressed as ios:<udid>

IOS_APPIUM_URL

Appium server URL, default http://127.0.0.1:4723

IOS_BUNDLE_ID

Default iOS app/browser bundle, e.g. com.google.chrome.ios or com.apple.mobilesafari

IOS_DEVICES_JSON

Per-device iOS config for multiple phones/simulators, WDA ports, bundle IDs, and MJPEG ports

IOS_MJPEG_SERVER_PORT

WDA MJPEG stream port; use one unique port per iOS device


Give any AI agent a mobile body (MCP)

Ghost ships an MCP server with 66 tools for real-device control. Any MCP-compatible client can use them. Android serials receive the Android implementation; ios:<udid> refs route to the iOS backend where supported and return stable unsupported-platform errors for Android-only tools. One command wires it up:

# Claude Code (same shape for Codex, Cursor, VS Code Copilot, Windsurf)
claude mcp add android-agent -- uvx --from ghost-in-the-droid android-agent-mcp

uvx installs the package, builds an isolated env, and runs the server. No clone, no venv.

# Codex (OpenAI)
codex mcp add android-agent -- uvx --from ghost-in-the-droid android-agent-mcp

Claude Desktop (claude_desktop_config.json), Cursor (.cursor/mcp.json), Windsurf (mcp_config.json):

{
  "mcpServers": {
    "android-agent": {
      "command": "uvx",
      "args": ["--from", "ghost-in-the-droid", "android-agent-mcp"]
    }
  }
}

VS Code Copilot (.vscode/mcp.json) uses the same block under a "servers" key. Contributors who clone the repo get a ready .mcp.json, all 66 tools live on first claude launch.

The 66 tools, by category:

Category

What the ghost can do

See

screenshot, get_elements, get_screen_tree, get_screen_xml, screenshot_annotated, screenshot_cropped

Touch

tap, tap_element, swipe, long_press, type_text, type_unicode, press_back, press_home, press_key

Apps

launch_app, search_apps, list_apps, launch_intent, force_stop, list_packages

Understand

get_phone_state, classify_screen, find_on_screen, ocr_screen, ocr_region, extract_visible_text

Browser / iOS

open_url, browser_back, get_current_url, read_news, extract_articles, wait_for_text

Device

list_devices, clipboard_get, clipboard_set, get_notifications, open_notifications, toggle_overlay, device_health

Skills

list_skills, run_workflow, run_action, create_skill, explore_app

Batch

run_flow / chain: one call runs a whole multi-step recipe server-side, N round-trips collapse to 1

Diagnostics

list_crashes, get_crash, web_search, screen recording, camera + TTS

toggle_overlay, launch_intent, Android shell helpers, wireless ADB, and Play Store helpers remain Android-only and return stable platform errors on iOS.


Run the brain inside the phone

New in 1.3: the model can live on the device. The Ghost companion app embeds real inference engines, so the phone is both the body and the brain. Airplane mode on, agent still working, nothing leaving the device.

Platform

Engine

Model format

Best for

Android

llama.cpp (JNI)

.gguf

Any GGUF: Gemma, Llama, Mistral, Qwen, DeepSeek

Android

MediaPipe

.task

Small Gemma models, fastest Android integration

iPhone

llama.cpp (Metal)

.gguf

Same GGUF as Android, Qwen2.5 1.5B on Metal

iPhone

MLX

Apple Silicon

Faster decode on newer chips, opt-in

On iPhone, Qwen2.5 1.5B runs through llama.cpp on Metal and drives the phone's own UI, with an opt-in MLX engine for faster decode on Apple Silicon. Small models are kept honest with grammar-constrained decoding so tool calls always parse. Full detail on the on-device LLM page.


Skill Hub CLI

Path: install, connect a phone, then your first hub skill:

ghost skill search calculator                # search the hub registry
ghost skill info ghost/calculator            # trust, quality tier, actions, tested-on device
ghost skill install ghost/calculator         # hash-checked install into ~/.ghost/skills
ghost skill run ghost/calculator calculate --param expression='1234*5' --device <serial>
ghost skill list                             # installed skills
ghost skill outdated                         # installed vs latest
ghost skill validate ./my-skill/             # before publishing
ghost skill submit ./skillhub/<you>/<slug> --push   # fork, push, open the PR

How a skill earns its tier and points is on How skills are scored; the submission flow is on Publishing Skills. The hub registry is skillhub/registry.json, regenerated by CI after merge.

Teach the Ghost a New App

Two ways to forge a skill:

Community skill (your own repo): create a repo with skill.yaml, elements.yaml, actions, and workflows, tag it android-agent-skill, and it appears on the Skill Hub automatically (nightly scraper).

Official skill (PR to this repo): build and test as a community skill first, then open a PR adding it to registry/. CI validates, a maintainer reviews, and it gets the "Official" badge.

Each skill needs:

  • skill.yaml: metadata (name, version, app package or iOS bundle ID, supported platforms, actions, workflows)

  • elements.yaml: Android UI element resource IDs and descriptions

  • elements_ios.yaml: optional iOS selectors for XCTest accessibility trees

  • actions/: Python classes extending Action with precondition() and execute()

  • workflows/: Python classes extending Workflow with steps()

Prefer to let AI do it? The Skill Creator watches a live device stream and generates the action and workflow code while you narrate. Full guide in CONTRIBUTING.md.


The dashboard

A local-first control room at http://localhost:5055 (or :6175 in dev). WebRTC and MJPEG live streams, tap-on-screen, tracing, and every tab below.

Tab

What it does

Phone Agent

Live device stream (MJPEG/WebRTC), tap/swipe on screen, multi-device view

Scheduler

Cron-style job scheduling with queue management

Skill Hub

Browse installed skills, run actions and workflows, export/delete

Skill Creator

LLM-assisted skill builder with live device stream

Skill Miner

Auto app explorer: BFS state discovery with screenshots

Manual Run

Start/stop bot jobs, queue management, logs

Tests

Per-device test runner with screen recording playback

Emulators

Create, boot, snapshot, and manage Android emulators

Tracing

Per-turn traces, token accounting, tool-call visibility


Architecture

android-agent/
  run.py                    # Uvicorn entry point on :5055
  gitd/
    app.py                  # FastAPI app factory + plugin hook
    models/  schemas/       # SQLAlchemy 2.0 ORM + Pydantic v2
    routers/  services/     # Route handlers + business logic
    skills/                 # Skill packages (tiktok, play_store, safari [iOS], tiktok_ios)
    bots/common/adb.py      # Android Device class: tap, swipe, dump tree, wait_for
    bots/common/ios.py      # iOS Device class: Appium/WDA session, UI tree, gestures
    mcp_server.py           # MCP server: 66 tools for any LLM agent
  frontend/                 # Vue 3 + Vite + TypeScript + Tailwind
  portal/                   # Kotlin companion app (WebRTC, on-device inference)
  site/                     # Docs site (Astro + Starlight)

The flow: FastAPI backend on :5055 exposes device control, skills, bots, scheduling, and streaming. The Vue SPA talks to it over /api/*. Skills define per-app interaction (elements + actions + workflows). Device backends route by ref: bare serials use bots/common/adb.py, ios:<udid> refs use bots/common/ios.py and Appium/WDA. State lives in SQLite via SQLAlchemy 2.0 and Alembic.

Benchmark (sneak peek)

Early result, with the full methodology writeup still cooking: driven by Claude Code, Ghost completes 115 of 116 tasks (99.1%) on AndroidWorld, Google Research's benchmark for Android agents, on the unmodified upstream harness. Treat this as a preview rather than a citable number. The detailed writeup and trajectories are coming. Watch the releases.


Tech Stack

Layer

Technology

Backend

FastAPI, Uvicorn, Python 3.10+

Data

SQLAlchemy 2.0, Alembic, Pydantic v2, SQLite (WAL)

Frontend

Vue 3, TypeScript, Vite, Tailwind CSS 4

Device control

ADB for Android; Appium XCUITest/WebDriverAgent for iOS

Streaming

Android MJPEG/WebRTC via Portal; iOS WDA MJPEG plus screenshot fallback

On-device

llama.cpp, MediaPipe (Android); llama.cpp on Metal, MLX (iOS)

Quality

Ruff, pytest, Playwright


Running Tests

Most tests are unit/API tests and run without a live device. Live Android and iOS integration tests require the relevant device stack.

# Run all tests on a specific device
DEVICE=<serial> python3 -m pytest tests/ -v --tb=short

For iOS live smoke tests:

IOS_LIVE_NEWS_TEST=1 \
IOS_DEVICE_UDID="<udid>" \
IOS_APPIUM_URL="http://127.0.0.1:4723" \
IOS_BUNDLE_ID="com.google.chrome.ios" \
uv run --extra test python -m pytest tests/test_browser_news.py::test_live_ios_chrome_news_workflow

Get your Android device serial from adb devices.


Database Migrations

The project uses Alembic for schema migrations:

alembic revision --autogenerate -m "add new_field to my_table"   # after editing a model
alembic upgrade head                                              # apply pending
alembic downgrade -1                                              # rollback one

Contributing

The ghost gets stronger with every skill. See CONTRIBUTING.md for adding app skills (highest impact), writing actions and workflows, backend architecture, and the PR process. Join the community on the Skill Hub and on Discord, where you can share the skills and things you build with Ghost and get help.


License

MIT. The ghost is free. The ghost is open source. The ghost is yours.

Available Tools

66 tools
app_stateA

Check whether an Android package or iOS bundle id is installed, running, or foreground.

Inputs: device (required), package (required). Returns: result

browser_backC

Navigate back in the current browser/app context.

Inputs: device (required). Returns: result

classify_screenA

Classify the current screen: what app, what type of screen (home, search, profile, settings, dialog, error, loading), keyboard state. No LLM needed — uses XML heuristics. Use this for quick state checks before deciding what action to take.

Inputs: device (required). Returns: result

clear_notificationsA

Dismiss visible notifications when the platform exposes a clear control.

Inputs: device (required). Returns: result

clipboard_getC

Get the current clipboard text from the device.

Inputs: device (required). Returns: result

clipboard_setA

Set clipboard text on the device. Use with press_key(PASTE) to paste into fields.

Inputs: text (required), device (required). Returns: result

create_skillA

Create a new reusable skill from a JSON list of recorded steps.

steps is a JSON array like: [ {"action": "launch", "package": "com.example.app", "description": "Open app"}, {"action": "tap", "x": 540, "y": 1200, "description": "Tap button"}, {"action": "type", "text": "hello", "description": "Type greeting"}, {"action": "wait", "seconds": 2, "description": "Wait for load"} ]

Supported actions: launch, tap (x,y or element_idx), type, swipe, back, home, wait. For iOS skills, pass platforms="ios" and either app_package or ios_bundle_id as the bundle id. Optional elements_ios/elements_android are JSON selector maps written to elements_ios.yaml/elements.yaml. After creating, use run_workflow(dev, name, "recorded", params) to replay it.

Inputs: name (required), steps (required), app_package (required), platforms, +3 more. Returns: result

crm_list_unread_messagesA

List local CRM contacts with unread messages, sorted by recency. Read-only.

Returns one row per unread conversation with the handle, unread count, last message preview, and timestamp.

Inputs: no named parameters. Returns: result

crm_lookup_contactA

Get the stored fact sheet for one local CRM contact by handle. Read-only.

Returns the contact's profile fields, contact status/history, and the latest conversation state (last message, unread count, timestamps).

Args: handle: Contact handle, with or without @.

Inputs: handle (required). Returns: result

device_healthC

Run a comprehensive device health check. iOS includes Appium/WDA status, active session details, and recovery steps.

Inputs: device (required). Returns: result

draft_skillA

Distil the device's most recent chat conversation into draft replayable steps for a HARD skill.

Returns the captured steps (with correct coords/args), a guessed app_package, and a summary. Nothing is written — review/prune/rename the steps, then call save_skill(kind="hard", steps=...).

Inputs: device (required). Returns: result

explore_appA

Explore an app's UI autonomously using BFS. Launches the app, taps every interactive element, builds a state graph. Returns JSON with discovered screens, elements, and transitions. Use this to understand an unfamiliar app before writing automation for it.

Inputs: device (required), package (required), max_depth, max_states. Returns: result

extract_articlesC

Extract likely visible article/headline candidates from the current browser page.

Inputs: device (required), max_items. Returns: result

extract_visible_textA

Extract visible text from the current screen with browser chrome filtered by default.

Inputs: device (required), max_lines, include_controls. Returns: result

find_on_screenA

Find specific text on the screen and return its location. Searches XML elements first (fast), falls back to OCR if not found. Use this to check if a button, label, or message is visible. Returns JSON with {text, x, y, w, h, method} or null if not found.

Inputs: text (required), device (required). Returns: result

fix_device_healthC

Apply a recovery action returned by device_health.recommended_fix.

Inputs: issue (required), device (required). Returns: result

force_stopA

Force-stop an Android package or terminate an iOS bundle id.

Inputs: device (required), package (required). Returns: result

get_crashA

Return the full stack trace of the MOST RECENT crash (from the crash buffer).

Android-only: iOS crash logs need syslog/CrashReporter access, not exposed yet. Optionally filter by package. Pair with list_crashes() to see what's there.

Inputs: device (required), package. Returns: result

get_current_urlC

Get the current browser URL when the platform exposes it.

Inputs: device (required). Returns: result

get_elementsB

Get all UI elements on the current screen as a JSON array. Each element has: idx, text, content_desc, resource_id, class, bounds, center, clickable, scrollable. Use element idx with tap_element(). Call this to understand the screen layout before acting.

Inputs: device (required), interactive_only. Returns: result

get_notificationsA

Get active notifications. Returns JSON array of {package, title, text}.

Inputs: device (required). Returns: result

get_phone_stateA

Get current app, activity, keyboard state, and focused element. Quick way to check what app/screen the device is on without parsing full elements.

Inputs: device (required). Returns: result

get_screen_treeA

Get an LLM-friendly indented UI hierarchy of the current screen. Each node shows: [idx] ClassName "label" [clickable] [x1,y1][x2,y2]. Use this to understand screen layout and pick which element to tap. Much more readable than raw XML — prefer this over get_elements() for planning.

Inputs: device (required). Returns: result

get_screen_xmlA

Get the raw normalized UI XML dump from the device. Android returns uiautomator XML; iOS returns normalized Appium/WDA XML. Use get_screen_tree() instead for a readable summary. Use this only when you need exact attribute values or the full hierarchy.

Inputs: device (required). Returns: result

get_stream_infoB

Return effective stream metadata without opening the stream.

iOS reports WDA MJPEG URL/settings, screenshot fallback, and unsupported Portal/WebRTC actions. Android reports Portal/H264/screencap metadata.

Inputs: device (required), fps, mode, quality. Returns: result

launch_appA

Launch an app by Android package name or iOS bundle id. Use search_apps() to find it.

Args: device: ADB serial or ios:. package: App package name or iOS bundle id, e.g. "com.android.chrome" or "com.google.chrome.ios". fresh: If True, force-stop the app first (cold start, clears in-memory state — back stack, unsaved drafts, login flow position, etc.). If False (default), reuses any existing background instance (warm start — resumes wherever the user left off). Use fresh=True for benchmarks, fresh start of a flow, or when the current app state would interfere with the task.

Inputs: device (required), package (required), fresh. Returns: result

launch_intentA

Launch a full Android intent. More powerful than launch_app(). Examples: Open a URL: action="android.intent.action.VIEW" data="https://google.com" Open Settings: package="com.android.settings" Share text: action="android.intent.action.SEND" extras='{"android.intent.extra.TEXT": "hello"}'

Inputs: device (required), data, action, extras, +1 more. Returns: result

list_appsB

List installed apps with human-readable names and package names or bundle ids. iOS is limited to configured/common bundle ids verified through Appium.

Inputs: device (required). Returns: result

list_crashesA

List recent app crashes and ANRs (from the logcat crash buffer, no root).

Android-only: iOS crash logs need syslog/CrashReporter access, not exposed yet. Optionally filter by package (substring match on the crashing process). Returns JSON: {count, crashes: [{type, timestamp, process, summary}]}. Use get_crash() to pull a full stack for the most recent one.

Inputs: device (required), limit, package. Returns: result

list_devicesA

List connected Android ADB devices and configured iOS Appium devices. Call this first to get the device serial you need for other tools.

Inputs: no named parameters. Returns: result

list_packagesA

List raw Android package names or iOS bundle ids. Prefer list_apps() for display names.

Inputs: device (required). Returns: result

list_skillsA

List all installed mobile automation skills with their actions, workflows, and platform support. Use this to discover what high-level automations are available. Prefer using run_workflow() over raw tap/swipe when a skill exists for the task.

Inputs: device, supported_only. Returns: result

list_unread_leadsA

List every influencer with an unread reply in the inbox, sorted by recency.

Returns one row per unread conversation with the handle, unread count, last message preview, and timestamp. Useful for daily prioritisation: 'which leads should I respond to right now?'

Inputs: no named parameters. Returns: result

long_pressB

Long press at coordinates. Use for context menus, drag initiation, etc.

Inputs: x (required), y (required), device (required), duration_ms. Returns: result

lookup_leadA

Get the full fact sheet for one influencer lead by handle.

Use this when you need to know everything about an influencer to draft a personalised reply or decide next-step outreach: their follower count, engagement, bio, niche, what hashtag we found them on, when we DMed them, which account sent the DM, their latest reply, and unread state.

Args: handle: TikTok username, with or without @ (e.g. 'creatorhandle' or '@creatorhandle')

Inputs: handle (required). Returns: result

ocr_regionA

OCR a specific region of the screen. Coordinates in device pixels. More accurate than full-screen OCR for targeted text extraction. Returns JSON array of {text, conf, x, y, w, h} relative to the crop region.

Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +1 more. Returns: result

ocr_screenA

OCR the entire device screen using RapidOCR. Returns all visible text with positions. Use this when UI elements are rendered as images/canvas (e.g., analytics dashboards, games, WebViews) where get_elements() returns no text. Returns JSON array of {text, conf, x, y, w, h} sorted top-to-bottom.

Inputs: device (required). Returns: result

open_cameraA

Open the platform camera app in a specific mode.

Android uses launcher/UI automation; iOS uses the Camera bundle and WDA UI controls. No package or bundle id is required.

Args: device: ADB serial or ios:. mode: One of: "photo" — rear camera, photo mode (default) "video" — rear camera, video/record mode "selfie" — front camera, photo mode "selfie_video" — front camera, video mode timer_s: Self-timer delay in seconds. Supported: 0 (off), 2, 3, 5, 10. Uses UI automation — snaps to the closest value the device supports (ASUS: 3s/10s, Samsung: 2s/5s/10s). 0 = no timer (default).

Inputs: device (required), mode, timer_s. Returns: result

open_notificationsC

Pull down the notification shade or iOS Notification Center.

Inputs: device (required). Returns: result

open_urlA

Open a URL in the platform browser.

On iOS this uses Appium/WDA and defaults to the configured browser bundle id, usually com.google.chrome.ios or com.apple.mobilesafari.

Inputs: url (required), device (required), bundle_id. Returns: result

paste_textA

Set clipboard text and immediately paste it into the currently focused field. Equivalent to clipboard_set + press_key(PASTE) in one call. Tap the target input field first to focus it, then call this.

Inputs: text (required), device (required). Returns: result

press_backC

Press Back on Android or the best available iOS browser/navigation back action.

Inputs: device (required). Returns: result

press_homeA

Press the platform Home button. Returns to the home screen.

Inputs: device (required). Returns: result

press_keyA

Send a platform key event. Android accepts KEYCODE_* names, with the KEYCODE_ prefix added automatically. iOS supports WDA-backed HOME, ENTER/RETURN, and BACK/ESCAPE.

Inputs: key (required), device (required). Returns: result

read_newsC

Open a news page and return structured headlines plus article snippets.

This is the iOS Chrome/WebDriver smoke workflow exposed as a single tool.

Inputs: device (required), url, wait_s, bundle_id, +3 more. Returns: result

run_actionA

Run a single skill action on the device.

Call list_skills() first to see available actions.

Examples: run_action("SERIAL", "tiktok", "open_app", '{}') run_action("SERIAL", "tiktok", "type_and_search", '{"query": "cats"}')

Inputs: skill (required), action (required), device (required), params. Returns: result

run_flowA

Run an ordered batch of tool calls server-side in ONE round-trip.

steps is a JSON list of {"tool": "", "args": {...}} — the same tool names execute_tool exposes (tap, tap_element, type_text, launch_app, etc.). Steps run in order and stop at the first error. Returns a JSON object with per-step results and a SINGLE final screenshot (not one per step) — far fewer tokens/round-trips than calling each tool separately. Steps do NOT auto-settle between UI actions; if a step needs the screen to update before the next one reads it, insert an explicit {"tool":"wait","args":{...}} step.

Example: run_flow("SERIAL", '[{"tool":"launch_app","args":{"package":"com.android.settings"}}, {"tool":"find_on_screen","args":{"text":"Wi-Fi"}}, {"tool":"tap","args":{"x":540,"y":300}}]')

Security: fail-closed allow-list — only vetted read/UI tools may run inside a flow. Anything else (raw shell, run_skill, or any unknown tool) makes the whole flow be refused before any step runs, since a batch is exactly where an injected instruction would smuggle a shell command. Max 50 steps.

Inputs: steps (required), device (required). Returns: result

run_workflowA

Run an installed skill workflow on the device.

Call list_skills() first to see available skills and workflows.

Examples: run_workflow("SERIAL", "tiktok", "upload_video", '{"video_path": "/tmp/video.mp4"}') run_workflow("SERIAL", "send_gmail_email", "recorded", '{"subject": "Hello", "body": "Test"}')

params is a JSON string of keyword arguments for the workflow.

Inputs: skill (required), device (required), workflow (required), params. Returns: result

save_skillA

Save the device's current chat conversation as a reusable skill.

kind="hard" replays concrete actions — pass revised steps (a JSON array from draft_skill), or omit them to auto-distil the conversation. kind="soft" stores markdown guidance (what to watch out for) surfaced to agents on demand via list_skills / run_workflow.

Inputs: name (required), device (required), kind, steps, +3 more. Returns: result

screen_recording_statusB

Return active screen recording status for a device.

Inputs: device (required). Returns: result

screenshotA

Take a screenshot of the device screen. Returns a base64-encoded JPEG. Use this to SEE what's on screen before deciding what to tap.

Routes through the shared compressed screenshot path (half-resolution JPEG, cross-platform) instead of a raw full-res PNG: a raw PNG base64 string overflows the MCP tool-result token cap on content-heavy screens, so the client falls back to text/OCR and never sees the pixels. The downscale cuts the payload ~4-8x so most screens stay under the cap. (The lasting fix is returning an MCP image-content block; tracked as feature #8.)

Inputs: device (required). Returns: result

screenshot_annotatedA

Take a screenshot with numbered element labels overlaid on interactive elements. The numbers correspond to element indices from get_elements(). Use this when you want to SEE the screen with elements visually labelled. Returns base64-encoded PNG.

Inputs: device (required). Returns: result

screenshot_croppedA

Take a screenshot of a specific region of the screen. Coordinates are in device pixels. Use this to zoom in on a specific area (e.g., a form field, a notification, a chart). Returns base64-encoded JPEG.

Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +1 more. Returns: result

search_appsA

Search installed apps by name. Case-insensitive. Returns Android packages or iOS bundle ids. Example: search_apps('tiktok') → [{"name": "TikTok", "package": "com.zhiliaoapp.musically"}]

Inputs: query (required), device (required). Returns: result

speak_textA

Make the phone speak text aloud using its built-in TTS engine.

Works whether the agent runs on the phone or on a PC — the call always goes through the Ghost portal app running on the device.

Args: device: ADB serial. This tool is Android-only. text: Text to speak. rate: Speech rate multiplier (0.5 = slow, 1.0 = normal, 1.5 = fast).

Inputs: text (required), device (required), rate. Returns: result

start_screen_recordingB

Start recording the device screen.

iOS uses WDA MJPEG captured through ffmpeg. Android uses adb screenrecord.

Inputs: device (required), filename. Returns: result

stop_screen_recordingC

Stop a running device screen recording and save the MP4.

Inputs: device (required). Returns: result

swipeA

Swipe from (x1,y1) to (x2,y2). Use for scrolling, pulling down notifications, etc. Common patterns: scroll down = swipe(dev, 540, 1400, 540, 600) scroll up = swipe(dev, 540, 600, 540, 1400)

Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +2 more. Returns: result

tapB

Tap at exact pixel coordinates (x, y) on the device screen.

Inputs: x (required), y (required), device (required). Returns: result

tap_elementA

Tap a UI element by its index from get_elements(). Call get_elements() first to see what's on screen and get element indices.

Inputs: idx (required), device (required). Returns: result

toggle_overlayA

Toggle the numbered element overlay on the device screen. When on, interactive elements get visible numbered labels that match get_elements() indices. Useful for visual debugging or when sending screenshots to a vision model.

Inputs: device (required), visible. Returns: result

type_textA

Type text into the currently focused input field. Tap an input field first to focus it. Spaces are supported. Non-ASCII input is transliterated to the closest ASCII (adb input text is ASCII-only); for full-fidelity emoji/CJK use type_unicode() instead.

Inputs: text (required), device (required). Returns: result

type_unicodeA

Type unicode text into the focused field. Android uses ADBKeyboard when configured; iOS uses WDA text entry. Use type_text() for plain ASCII.

Inputs: text (required), device (required). Returns: result

waitB

Pause execution for a fixed number of seconds.

Inputs: seconds. Returns: result

wait_for_textC

Wait until text appears on screen and return visible text context.

Inputs: text (required), device (required), timeout. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3/5.0

    Scored across 66 tools

    Disambiguation2/5

    Many tools have overlapping purposes (e.g., tap_element vs tap, get_elements vs get_screen_tree vs get_screen_xml, screenshot variants, OCR variants, app listing tools, and CRM lookup tools). Even with descriptions, an agent may struggle to select the correct tool.

    Naming Consistency3/5

    Mostly snake_case verb_noun, but there are bare verbs (swipe, tap, wait), noun-only names (device_health, app_state, screen_recording_status), and inconsistent prefix usage (crm_lookup_contact vs lookup_lead). The pattern is readable but not fully consistent.

    Tool Count1/5

    66 tools is excessive for a single server, far beyond the typical 3-15 well-scoped set. This causes usability issues despite the broad domain.

    Completeness3/5

    The server covers many mobile automation areas (UI, OCR, workflows, skills, crashes, CRM), but notable gaps remain—no direct app uninstall, no lead update/reply, no device setting controls. Agents can work around some gaps via workflows.

    Maintenance

    ActivityMaintained
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      An MCP server that lets AI agents control iOS and Android devices (tap, scroll, type, take screenshots, read UI trees, and run code). Works with multiple devices at the same time.
      91 npm
      53
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables AI agents to control Android devices via ADB, providing tools for screen capture, tap/swipe/input, app management, and diagnostics. Works locally over USB or WiFi and integrates with MCP clients like Claude and Cursor.
      1
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Controls Android and iOS devices from AI agents through a token-efficient MCP server, enabling observation, tapping, typing, gestures, navigation, assertions, and safety-gated actions.
      2
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables AI agents to control a physical Android device or Docker-hosted emulator over ADB by taking screenshots and performing taps, swipes, key presses, text input, shell commands, app launches, and device info queries. It runs as a local MCP stdio server alongside a web UI and REST API for remote access.
      1
      MIT