ghost-in-the-droid
OfficialThis MCP server lets an AI agent control real Android and iOS phones and automate mobile workflows through device, screen, app, browser, skill, diagnostic, and CRM tools.
Discover and inspect devices: list Android ADB/iOS Appium devices, run health checks, and apply recovery fixes.
See the screen: take full, annotated, or cropped screenshots; inspect UI elements, screen trees/XML, OCR text, screen type, and phone/app state.
Interact with the phone: tap, tap elements, swipe, long-press, type ASCII/unicode text, paste, press keys/back/home, and wait.
Control apps: launch, force-stop, check app state, list/search installed apps or packages, launch Android intents, open camera, and use Android TTS.
Browse the web: open URLs, run web searches, navigate back, get current URL, wait for text, extract visible text/articles, and read news.
Use device utilities: manage clipboard, read/open/clear notifications, record screen, inspect stream info, and toggle element overlays.
Automate with skills: list/run workflows and actions, create/draft/save reusable skills, explore apps with BFS, and run batched tool flows server-side.
Diagnose apps: list crashes/ANRs and fetch crash stack traces on Android.
Query lead/CRM data: look up influencer leads or CRM contacts and list unread leads/messages.
Controls Android devices over ADB, enabling taps, swipes, text input, screen capture, app launching, and shell commands.
Demonstrates automated control of Duolingo lessons on a real device.
Controls iOS devices via Appium/WebDriverAgent, enabling taps, swipes, text input, screen capture, and app automation.
Includes built-in skills for automating TikTok workflows, such as browsing and interacting with content.
The pitch, in one breath
Every AI agent can think. Almost none can touch a phone.
Ghost is the body. It gives any LLM agent a real Android phone or iPhone: it reads the screen, taps, swipes, and types through a clean tool surface, and it scales from one device on your desk to a whole phone farm. Point Claude Code, Codex, Antigravity, Cursor, your LangChain app, or a model running inside the phone itself at it, and your agent grows hands.
Bring your own brain. Keep the same body. MIT, forever.
Related MCP server: adb-mcp
Mix and match: the Ghost matrix
Ghost 1.3 is the only Android + iOS agent framework where platform, brain, and driver are all swappable. Pick one from each column. They all compose.
Pick a platform | Pick a brain | Pick a driver |
Android over USB ADB | Claude Code (free with your Max/Pro sub) | MCP client (Claude Code, Codex, Antigravity, Cursor, Opencode) |
Android over Wi-Fi ADB | Anthropic API | LangChain toolkit |
Android in a Docker + KVM emulator pool | OpenRouter (any model, one key) | LlamaIndex tool spec |
iPhone over Appium + WebDriverAgent | Ollama (fully local) | Ghost CLI ( |
iPhone over Tailscale + WebDriverAgent (wireless) | vLLM (your own GPU) | Web dashboard chat with a live phone stream |
On-device — llama.cpp ( | REST API ( |
The Docker + KVM emulator pool is Android-only; iOS uses Apple's own simulator and Xcode signing, which are macOS-native.
The body stays the same 66 MCP tools no matter what you plug in. Every new model release is a free upgrade to your phone agent.
See it in action
Nine demos, one per feature, each recorded on real devices. Click any clip for the full mp4.
What It Does
Open-source Python framework for controlling Android and iOS devices from one agent harness. Android runs through ADB. iOS runs through Appium XCUITest and WebDriverAgent, with real iPhones as the target path and simulators for development and CI.
Define skills for any app, run them from the dashboard or API, scale across a phone farm.
The ghost taps what you'd tap
Android control through ADB: tap, swipe, type, clipboard, shell, intents, and stealth variants
iOS control through WebDriverAgent: screenshot, accessibility tree, tap, swipe, type, app launch, clipboard, browser actions
Live phone screen streaming: Android MJPEG/WebRTC, iOS WDA MJPEG with screenshot fallback
Interactive touch-to-tap on the streamed screen
Multi-device phone farm with per-device job queues
Forge reusable skills for any app
YAML-based UI element definitions per app
Platform-specific selectors with
elements.yamlfor Android andelements_ios.yamlfor iOSPython action classes with precondition checks
Multi-step workflows that chain actions together
Built-in skills for TikTok and Play Store
iOS browser/news demo skill and smoke-level TikTok iOS workflows
Skill Hub: browse, search, and install skills from the community registry
Install from CLI:
android-agent skill install tiktok
Teach the ghost new tricks
BFS-based auto app explorer: discovers every screen and transition
LLM-assisted Skill Creator: chat with AI while viewing the live device stream
The AI identifies UI elements and generates action/workflow code
Scale the haunting
Multi-device phone farm with per-device job queues
Bot runner: queue, schedule, and monitor automation jobs
Per-device integration tests with Android screen recording or iOS WDA MJPEG recording
Platform Support
Surface | Android | iOS |
Device ref | ADB serial, e.g. |
|
Backend | ADB + optional Portal companion app | Appium XCUITest + WebDriverAgent |
Real device support | Yes | Yes, with Mac/Xcode/WDA signing |
Simulator/emulator support | Android emulator tooling | Booted iOS simulators via Appium/WDA |
Live stream | Portal WebRTC, MJPEG, screencap | WDA MJPEG, screenshot polling fallback |
Screen tree | Android UIAutomator XML | Normalized XCTest accessibility tree |
Skills |
|
|
Android-only today | ADB shell, intents, wireless ADB, Play Store helpers, Portal overlay | Unsupported with stable platform errors |
Why it is built this way
Skills cost $0. Thinking costs tokens. An agent should not pay an LLM to tap a button it has tapped a thousand times. Known workflows compile to skills: deterministic YAML + Python recipes that replay in seconds with zero LLM calls. Use AI for the unknown task, replay for the known one.
The phone is the easiest sandbox there is. Give the ghost an old Android with its own SIM and its own accounts, physically separate from your life. You already know how to set up a phone.
Zero-app by default. A fresh install touches nothing on the Android device: screen reads go through uiautomator, actions through adb shell input, no root, no accessibility service. Want it faster? The optional Portal companion app gives roughly 30x quicker UI reads. Want it private? On-device mode runs the whole loop inside the phone, nothing leaves it. All opt-in.
Local-first, cloud-optional. The server and dashboard run on your machine. With a local or on-device brain, your screenshots never leave your network. Pick a cloud brain and prompts go to that provider, same as any tool. Your call, always.
Requirements
Python 3.10+
Android: Android phone with USB debugging enabled and ADB on PATH (
adb devicesshould list your phone)iOS: macOS with Xcode, Appium 2 + XCUITest driver, and a trusted iPhone or booted simulator
Node.js 18+ (for the frontend dev server)
iOS Support
Ghost drives iPhones through Appium/WebDriverAgent with the same tool
surface: ios:<udid> device refs route tap/swipe/type/screenshot to WDA, and
iOS-aware browser primitives (open_url, read_news, extract_visible_text)
cover web tasks. Android-only tools (shell, launch_intent, Portal overlay)
return a clear platform error instead of failing silently.
iOS is opt-in: enable it with GITD_ENABLE_IOS=1 (or ios_platform_enabled=true
in .env). Wireless drive rides your Tailscale tailnet, so treat the tailnet as
the trust boundary. See docs/SETUP_IOS.md for Appium/WDA
setup, and on-device on iPhone
to run the model on the iPhone itself.
Quick Start
Zero-install with uvx (or pipx):
uvx ghost-in-the-droid doctor # green/red preflight: python, adb on PATH, devices, ports, LLM keys
uvx ghost-in-the-droid login # sign in with your Claude subscription, no API key needed
uvx ghost-in-the-droid up # server + dashboard at http://localhost:5055 (API docs at /docs)doctor prints a checklist with fix hints instead of a stack trace when something is missing (e.g. adb not on PATH). Prefer pipx? pipx install ghost-in-the-droid gives you the ghost-in-the-droid / android-agent commands.
Sign in with your Claude subscription (no API key)
If you have a Claude Max/Pro subscription, you don't need an API key. android-agent login signs you in through the claude CLI's own Anthropic OAuth flow and points Ghost at the claude-code provider:
android-agent login # opens Anthropic sign-in via the claude CLIGhost never handles or stores your token; the claude CLI owns it, refresh included. doctor shows a green Claude subscription check once you are signed in. To use an API key instead, set ANTHROPIC_API_KEY (or OPENAI_API_KEY / OPENROUTER_API_KEY) and pick that provider.
Select Ollama in the Phone Agent tab. Runs entirely on your machine with Ollama:
brew install ollama # or: curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull llama3.2:3b # 2GB, fast, good tool-usegit clone https://github.com/ghost-in-the-droid/android-agent.git
cd android-agent
pip install -e ".[all]"
android-agent doctor # preflight
android-agent up # start server + dashboard on :5055
# Frontend (separate terminal)
cd frontend && npm install && npx vite --host 0.0.0.0 --port 6175
# Dashboard at http://localhost:6175iOS Quick Start
iOS support requires a Mac because Appium uses Xcode's XCUITest/WebDriverAgent stack. Real iPhones also require trust, Developer Mode, UI Automation permission when prompted, and WDA signing with an Apple development team.
# 1. Install and run Appium XCUITest
npm install -g appium
appium driver install xcuitest
appium --base-path /
# 2. Find your iPhone or booted simulator UDID
xcrun xctrace list devices
xcrun simctl list devices booted
# 3. Configure the backend for iOS
export IOS_DEVICE_UDID="<udid>"
export IOS_APPIUM_URL="http://127.0.0.1:4723"
export IOS_BUNDLE_ID="com.google.chrome.ios" # or com.apple.mobilesafari
export IOS_MJPEG_SERVER_PORT="9100" # use unique ports per iOS device
# 4. Run a product-path smoke workflow
uv run python scripts/ios_chrome_news_smoke.py \
--device "ios:<udid>" \
--bundle-id "$IOS_BUNDLE_ID" \
--url https://text.npr.org/ \
--max-headlines 5 \
--max-articles 3 \
--fix-health \
--out-dir data/ios_chrome_news_smokeFor full real-device signing, simulator, WDA MJPEG, health recovery, scheduler, and MCP setup details, see docs/SETUP_IOS.md.
Environment Variables
Copy .env.example to .env (if provided) or create a .env file in the project root. The server reads configuration via Pydantic Settings. Optional variables include:
Variable | Purpose |
| LLM features (Skill Creator, Agent Chat) |
| Alternative LLM provider |
| OpenRouter LLM provider |
| ADB serial (auto-detected if empty) |
| iPhone or simulator UDID; devices are addressed as |
| Appium server URL, default |
| Default iOS app/browser bundle, e.g. |
| Per-device iOS config for multiple phones/simulators, WDA ports, bundle IDs, and MJPEG ports |
| WDA MJPEG stream port; use one unique port per iOS device |
Give any AI agent a mobile body (MCP)
Ghost ships an MCP server with 66 tools for real-device control. Any MCP-compatible client can use them. Android serials receive the Android implementation; ios:<udid> refs route to the iOS backend where supported and return stable unsupported-platform errors for Android-only tools. One command wires it up:
# Claude Code (same shape for Codex, Cursor, VS Code Copilot, Windsurf)
claude mcp add android-agent -- uvx --from ghost-in-the-droid android-agent-mcpuvx installs the package, builds an isolated env, and runs the server. No clone, no venv.
# Codex (OpenAI)
codex mcp add android-agent -- uvx --from ghost-in-the-droid android-agent-mcpClaude Desktop (claude_desktop_config.json), Cursor (.cursor/mcp.json), Windsurf (mcp_config.json):
{
"mcpServers": {
"android-agent": {
"command": "uvx",
"args": ["--from", "ghost-in-the-droid", "android-agent-mcp"]
}
}
}VS Code Copilot (.vscode/mcp.json) uses the same block under a "servers" key. Contributors who clone the repo get a ready .mcp.json, all 66 tools live on first claude launch.
The 66 tools, by category:
Category | What the ghost can do |
See |
|
Touch |
|
Apps |
|
Understand |
|
Browser / iOS |
|
Device |
|
Skills |
|
Batch |
|
Diagnostics |
|
toggle_overlay, launch_intent, Android shell helpers, wireless ADB, and Play Store helpers remain Android-only and return stable platform errors on iOS.
Run the brain inside the phone
New in 1.3: the model can live on the device. The Ghost companion app embeds real inference engines, so the phone is both the body and the brain. Airplane mode on, agent still working, nothing leaving the device.
Platform | Engine | Model format | Best for |
Android | llama.cpp (JNI) |
| Any GGUF: Gemma, Llama, Mistral, Qwen, DeepSeek |
Android | MediaPipe |
| Small Gemma models, fastest Android integration |
iPhone | llama.cpp (Metal) |
| Same GGUF as Android, Qwen2.5 1.5B on Metal |
iPhone | MLX | Apple Silicon | Faster decode on newer chips, opt-in |
On iPhone, Qwen2.5 1.5B runs through llama.cpp on Metal and drives the phone's own UI, with an opt-in MLX engine for faster decode on Apple Silicon. Small models are kept honest with grammar-constrained decoding so tool calls always parse. Full detail on the on-device LLM page.
Skill Hub CLI
Path: install, connect a phone, then your first hub skill:
ghost skill search calculator # search the hub registry
ghost skill info ghost/calculator # trust, quality tier, actions, tested-on device
ghost skill install ghost/calculator # hash-checked install into ~/.ghost/skills
ghost skill run ghost/calculator calculate --param expression='1234*5' --device <serial>
ghost skill list # installed skills
ghost skill outdated # installed vs latest
ghost skill validate ./my-skill/ # before publishing
ghost skill submit ./skillhub/<you>/<slug> --push # fork, push, open the PRHow a skill earns its tier and points is on How skills are scored; the submission flow is on Publishing Skills. The hub registry is skillhub/registry.json, regenerated by CI after merge.
Teach the Ghost a New App
Two ways to forge a skill:
Community skill (your own repo): create a repo with skill.yaml, elements.yaml, actions, and workflows, tag it android-agent-skill, and it appears on the Skill Hub automatically (nightly scraper).
Official skill (PR to this repo): build and test as a community skill first, then open a PR adding it to registry/. CI validates, a maintainer reviews, and it gets the "Official" badge.
Each skill needs:
skill.yaml: metadata (name, version, app package or iOS bundle ID, supported platforms, actions, workflows)elements.yaml: Android UI element resource IDs and descriptionselements_ios.yaml: optional iOS selectors for XCTest accessibility treesactions/: Python classes extendingActionwithprecondition()andexecute()workflows/: Python classes extendingWorkflowwithsteps()
Prefer to let AI do it? The Skill Creator watches a live device stream and generates the action and workflow code while you narrate. Full guide in CONTRIBUTING.md.
The dashboard
A local-first control room at http://localhost:5055 (or :6175 in dev). WebRTC and MJPEG live streams, tap-on-screen, tracing, and every tab below.
Tab | What it does |
Phone Agent | Live device stream (MJPEG/WebRTC), tap/swipe on screen, multi-device view |
Scheduler | Cron-style job scheduling with queue management |
Skill Hub | Browse installed skills, run actions and workflows, export/delete |
Skill Creator | LLM-assisted skill builder with live device stream |
Skill Miner | Auto app explorer: BFS state discovery with screenshots |
Manual Run | Start/stop bot jobs, queue management, logs |
Tests | Per-device test runner with screen recording playback |
Emulators | Create, boot, snapshot, and manage Android emulators |
Tracing | Per-turn traces, token accounting, tool-call visibility |
Architecture
android-agent/
run.py # Uvicorn entry point on :5055
gitd/
app.py # FastAPI app factory + plugin hook
models/ schemas/ # SQLAlchemy 2.0 ORM + Pydantic v2
routers/ services/ # Route handlers + business logic
skills/ # Skill packages (tiktok, play_store, safari [iOS], tiktok_ios)
bots/common/adb.py # Android Device class: tap, swipe, dump tree, wait_for
bots/common/ios.py # iOS Device class: Appium/WDA session, UI tree, gestures
mcp_server.py # MCP server: 66 tools for any LLM agent
frontend/ # Vue 3 + Vite + TypeScript + Tailwind
portal/ # Kotlin companion app (WebRTC, on-device inference)
site/ # Docs site (Astro + Starlight)The flow: FastAPI backend on :5055 exposes device control, skills, bots, scheduling, and streaming. The Vue SPA talks to it over /api/*. Skills define per-app interaction (elements + actions + workflows). Device backends route by ref: bare serials use bots/common/adb.py, ios:<udid> refs use bots/common/ios.py and Appium/WDA. State lives in SQLite via SQLAlchemy 2.0 and Alembic.
Benchmark (sneak peek)
Early result, with the full methodology writeup still cooking: driven by Claude Code, Ghost completes 115 of 116 tasks (99.1%) on AndroidWorld, Google Research's benchmark for Android agents, on the unmodified upstream harness. Treat this as a preview rather than a citable number. The detailed writeup and trajectories are coming. Watch the releases.
Tech Stack
Layer | Technology |
Backend | FastAPI, Uvicorn, Python 3.10+ |
Data | SQLAlchemy 2.0, Alembic, Pydantic v2, SQLite (WAL) |
Frontend | Vue 3, TypeScript, Vite, Tailwind CSS 4 |
Device control | ADB for Android; Appium XCUITest/WebDriverAgent for iOS |
Streaming | Android MJPEG/WebRTC via Portal; iOS WDA MJPEG plus screenshot fallback |
On-device | llama.cpp, MediaPipe (Android); llama.cpp on Metal, MLX (iOS) |
Quality | Ruff, pytest, Playwright |
Running Tests
Most tests are unit/API tests and run without a live device. Live Android and iOS integration tests require the relevant device stack.
# Run all tests on a specific device
DEVICE=<serial> python3 -m pytest tests/ -v --tb=shortFor iOS live smoke tests:
IOS_LIVE_NEWS_TEST=1 \
IOS_DEVICE_UDID="<udid>" \
IOS_APPIUM_URL="http://127.0.0.1:4723" \
IOS_BUNDLE_ID="com.google.chrome.ios" \
uv run --extra test python -m pytest tests/test_browser_news.py::test_live_ios_chrome_news_workflowGet your Android device serial from adb devices.
Database Migrations
The project uses Alembic for schema migrations:
alembic revision --autogenerate -m "add new_field to my_table" # after editing a model
alembic upgrade head # apply pending
alembic downgrade -1 # rollback oneContributing
The ghost gets stronger with every skill. See CONTRIBUTING.md for adding app skills (highest impact), writing actions and workflows, backend architecture, and the PR process. Join the community on the Skill Hub and on Discord, where you can share the skills and things you build with Ghost and get help.
License
MIT. The ghost is free. The ghost is open source. The ghost is yours.
Available Tools
66 toolsapp_stateA
Check whether an Android package or iOS bundle id is installed, running, or foreground.
Inputs: device (required), package (required). Returns: result
browser_backC
Navigate back in the current browser/app context.
Inputs: device (required). Returns: result
classify_screenA
Classify the current screen: what app, what type of screen (home, search, profile, settings, dialog, error, loading), keyboard state. No LLM needed — uses XML heuristics. Use this for quick state checks before deciding what action to take.
Inputs: device (required). Returns: result
clear_notificationsA
Dismiss visible notifications when the platform exposes a clear control.
Inputs: device (required). Returns: result
clipboard_getC
Get the current clipboard text from the device.
Inputs: device (required). Returns: result
clipboard_setA
Set clipboard text on the device. Use with press_key(PASTE) to paste into fields.
Inputs: text (required), device (required). Returns: result
create_skillA
Create a new reusable skill from a JSON list of recorded steps.
steps is a JSON array like: [ {"action": "launch", "package": "com.example.app", "description": "Open app"}, {"action": "tap", "x": 540, "y": 1200, "description": "Tap button"}, {"action": "type", "text": "hello", "description": "Type greeting"}, {"action": "wait", "seconds": 2, "description": "Wait for load"} ]
Supported actions: launch, tap (x,y or element_idx), type, swipe, back, home, wait. For iOS skills, pass platforms="ios" and either app_package or ios_bundle_id as the bundle id. Optional elements_ios/elements_android are JSON selector maps written to elements_ios.yaml/elements.yaml. After creating, use run_workflow(dev, name, "recorded", params) to replay it.
Inputs: name (required), steps (required), app_package (required), platforms, +3 more. Returns: result
crm_list_unread_messagesA
List local CRM contacts with unread messages, sorted by recency. Read-only.
Returns one row per unread conversation with the handle, unread count, last message preview, and timestamp.
Inputs: no named parameters. Returns: result
crm_lookup_contactA
Get the stored fact sheet for one local CRM contact by handle. Read-only.
Returns the contact's profile fields, contact status/history, and the latest conversation state (last message, unread count, timestamps).
Args: handle: Contact handle, with or without @.
Inputs: handle (required). Returns: result
device_healthC
Run a comprehensive device health check. iOS includes Appium/WDA status, active session details, and recovery steps.
Inputs: device (required). Returns: result
draft_skillA
Distil the device's most recent chat conversation into draft replayable steps for a HARD skill.
Returns the captured steps (with correct coords/args), a guessed app_package, and a summary. Nothing is written — review/prune/rename the steps, then call save_skill(kind="hard", steps=...).
Inputs: device (required). Returns: result
explore_appA
Explore an app's UI autonomously using BFS. Launches the app, taps every interactive element, builds a state graph. Returns JSON with discovered screens, elements, and transitions. Use this to understand an unfamiliar app before writing automation for it.
Inputs: device (required), package (required), max_depth, max_states. Returns: result
extract_articlesC
Extract likely visible article/headline candidates from the current browser page.
Inputs: device (required), max_items. Returns: result
extract_visible_textA
Extract visible text from the current screen with browser chrome filtered by default.
Inputs: device (required), max_lines, include_controls. Returns: result
find_on_screenA
Find specific text on the screen and return its location. Searches XML elements first (fast), falls back to OCR if not found. Use this to check if a button, label, or message is visible. Returns JSON with {text, x, y, w, h, method} or null if not found.
Inputs: text (required), device (required). Returns: result
fix_device_healthC
Apply a recovery action returned by device_health.recommended_fix.
Inputs: issue (required), device (required). Returns: result
force_stopA
Force-stop an Android package or terminate an iOS bundle id.
Inputs: device (required), package (required). Returns: result
get_crashA
Return the full stack trace of the MOST RECENT crash (from the crash buffer).
Android-only: iOS crash logs need syslog/CrashReporter access, not exposed yet. Optionally filter by package. Pair with list_crashes() to see what's there.
Inputs: device (required), package. Returns: result
get_current_urlC
Get the current browser URL when the platform exposes it.
Inputs: device (required). Returns: result
get_elementsB
Get all UI elements on the current screen as a JSON array. Each element has: idx, text, content_desc, resource_id, class, bounds, center, clickable, scrollable. Use element idx with tap_element(). Call this to understand the screen layout before acting.
Inputs: device (required), interactive_only. Returns: result
get_notificationsA
Get active notifications. Returns JSON array of {package, title, text}.
Inputs: device (required). Returns: result
get_phone_stateA
Get current app, activity, keyboard state, and focused element. Quick way to check what app/screen the device is on without parsing full elements.
Inputs: device (required). Returns: result
get_screen_treeA
Get an LLM-friendly indented UI hierarchy of the current screen. Each node shows: [idx] ClassName "label" [clickable] [x1,y1][x2,y2]. Use this to understand screen layout and pick which element to tap. Much more readable than raw XML — prefer this over get_elements() for planning.
Inputs: device (required). Returns: result
get_screen_xmlA
Get the raw normalized UI XML dump from the device. Android returns uiautomator XML; iOS returns normalized Appium/WDA XML. Use get_screen_tree() instead for a readable summary. Use this only when you need exact attribute values or the full hierarchy.
Inputs: device (required). Returns: result
get_stream_infoB
Return effective stream metadata without opening the stream.
iOS reports WDA MJPEG URL/settings, screenshot fallback, and unsupported Portal/WebRTC actions. Android reports Portal/H264/screencap metadata.
Inputs: device (required), fps, mode, quality. Returns: result
launch_appA
Launch an app by Android package name or iOS bundle id. Use search_apps() to find it.
Args: device: ADB serial or ios:. package: App package name or iOS bundle id, e.g. "com.android.chrome" or "com.google.chrome.ios". fresh: If True, force-stop the app first (cold start, clears in-memory state — back stack, unsaved drafts, login flow position, etc.). If False (default), reuses any existing background instance (warm start — resumes wherever the user left off). Use fresh=True for benchmarks, fresh start of a flow, or when the current app state would interfere with the task.
Inputs: device (required), package (required), fresh. Returns: result
launch_intentA
Launch a full Android intent. More powerful than launch_app(). Examples: Open a URL: action="android.intent.action.VIEW" data="https://google.com" Open Settings: package="com.android.settings" Share text: action="android.intent.action.SEND" extras='{"android.intent.extra.TEXT": "hello"}'
Inputs: device (required), data, action, extras, +1 more. Returns: result
list_appsB
List installed apps with human-readable names and package names or bundle ids. iOS is limited to configured/common bundle ids verified through Appium.
Inputs: device (required). Returns: result
list_crashesA
List recent app crashes and ANRs (from the logcat crash buffer, no root).
Android-only: iOS crash logs need syslog/CrashReporter access, not exposed yet. Optionally filter by package (substring match on the crashing process). Returns JSON: {count, crashes: [{type, timestamp, process, summary}]}. Use get_crash() to pull a full stack for the most recent one.
Inputs: device (required), limit, package. Returns: result
list_devicesA
List connected Android ADB devices and configured iOS Appium devices. Call this first to get the device serial you need for other tools.
Inputs: no named parameters. Returns: result
list_packagesA
List raw Android package names or iOS bundle ids. Prefer list_apps() for display names.
Inputs: device (required). Returns: result
list_skillsA
List all installed mobile automation skills with their actions, workflows, and platform support. Use this to discover what high-level automations are available. Prefer using run_workflow() over raw tap/swipe when a skill exists for the task.
Inputs: device, supported_only. Returns: result
list_unread_leadsA
List every influencer with an unread reply in the inbox, sorted by recency.
Returns one row per unread conversation with the handle, unread count, last message preview, and timestamp. Useful for daily prioritisation: 'which leads should I respond to right now?'
Inputs: no named parameters. Returns: result
long_pressB
Long press at coordinates. Use for context menus, drag initiation, etc.
Inputs: x (required), y (required), device (required), duration_ms. Returns: result
lookup_leadA
Get the full fact sheet for one influencer lead by handle.
Use this when you need to know everything about an influencer to draft a personalised reply or decide next-step outreach: their follower count, engagement, bio, niche, what hashtag we found them on, when we DMed them, which account sent the DM, their latest reply, and unread state.
Args: handle: TikTok username, with or without @ (e.g. 'creatorhandle' or '@creatorhandle')
Inputs: handle (required). Returns: result
ocr_regionA
OCR a specific region of the screen. Coordinates in device pixels. More accurate than full-screen OCR for targeted text extraction. Returns JSON array of {text, conf, x, y, w, h} relative to the crop region.
Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +1 more. Returns: result
ocr_screenA
OCR the entire device screen using RapidOCR. Returns all visible text with positions. Use this when UI elements are rendered as images/canvas (e.g., analytics dashboards, games, WebViews) where get_elements() returns no text. Returns JSON array of {text, conf, x, y, w, h} sorted top-to-bottom.
Inputs: device (required). Returns: result
open_cameraA
Open the platform camera app in a specific mode.
Android uses launcher/UI automation; iOS uses the Camera bundle and WDA UI controls. No package or bundle id is required.
Args: device: ADB serial or ios:. mode: One of: "photo" — rear camera, photo mode (default) "video" — rear camera, video/record mode "selfie" — front camera, photo mode "selfie_video" — front camera, video mode timer_s: Self-timer delay in seconds. Supported: 0 (off), 2, 3, 5, 10. Uses UI automation — snaps to the closest value the device supports (ASUS: 3s/10s, Samsung: 2s/5s/10s). 0 = no timer (default).
Inputs: device (required), mode, timer_s. Returns: result
open_notificationsC
Pull down the notification shade or iOS Notification Center.
Inputs: device (required). Returns: result
open_urlA
Open a URL in the platform browser.
On iOS this uses Appium/WDA and defaults to the configured browser bundle id, usually com.google.chrome.ios or com.apple.mobilesafari.
Inputs: url (required), device (required), bundle_id. Returns: result
paste_textA
Set clipboard text and immediately paste it into the currently focused field. Equivalent to clipboard_set + press_key(PASTE) in one call. Tap the target input field first to focus it, then call this.
Inputs: text (required), device (required). Returns: result
press_backC
Press Back on Android or the best available iOS browser/navigation back action.
Inputs: device (required). Returns: result
press_homeA
Press the platform Home button. Returns to the home screen.
Inputs: device (required). Returns: result
press_keyA
Send a platform key event. Android accepts KEYCODE_* names, with the KEYCODE_ prefix added automatically. iOS supports WDA-backed HOME, ENTER/RETURN, and BACK/ESCAPE.
Inputs: key (required), device (required). Returns: result
read_newsC
Open a news page and return structured headlines plus article snippets.
This is the iOS Chrome/WebDriver smoke workflow exposed as a single tool.
Inputs: device (required), url, wait_s, bundle_id, +3 more. Returns: result
run_actionA
Run a single skill action on the device.
Call list_skills() first to see available actions.
Examples: run_action("SERIAL", "tiktok", "open_app", '{}') run_action("SERIAL", "tiktok", "type_and_search", '{"query": "cats"}')
Inputs: skill (required), action (required), device (required), params. Returns: result
run_flowA
Run an ordered batch of tool calls server-side in ONE round-trip.
steps is a JSON list of {"tool": "", "args": {...}} — the same tool
names execute_tool exposes (tap, tap_element, type_text, launch_app, etc.).
Steps run in order and stop at the first error. Returns a JSON object with
per-step results and a SINGLE final screenshot (not one per step) — far
fewer tokens/round-trips than calling each tool separately. Steps do NOT
auto-settle between UI actions; if a step needs the screen to update before
the next one reads it, insert an explicit {"tool":"wait","args":{...}} step.
Example: run_flow("SERIAL", '[{"tool":"launch_app","args":{"package":"com.android.settings"}}, {"tool":"find_on_screen","args":{"text":"Wi-Fi"}}, {"tool":"tap","args":{"x":540,"y":300}}]')
Security: fail-closed allow-list — only vetted read/UI tools may run inside
a flow. Anything else (raw shell, run_skill, or any unknown tool) makes
the whole flow be refused before any step runs, since a batch is exactly
where an injected instruction would smuggle a shell command. Max 50 steps.
Inputs: steps (required), device (required). Returns: result
run_workflowA
Run an installed skill workflow on the device.
Call list_skills() first to see available skills and workflows.
Examples: run_workflow("SERIAL", "tiktok", "upload_video", '{"video_path": "/tmp/video.mp4"}') run_workflow("SERIAL", "send_gmail_email", "recorded", '{"subject": "Hello", "body": "Test"}')
params is a JSON string of keyword arguments for the workflow.
Inputs: skill (required), device (required), workflow (required), params. Returns: result
save_skillA
Save the device's current chat conversation as a reusable skill.
kind="hard" replays concrete actions — pass revised steps (a JSON array from draft_skill), or
omit them to auto-distil the conversation. kind="soft" stores markdown guidance (what to watch
out for) surfaced to agents on demand via list_skills / run_workflow.
Inputs: name (required), device (required), kind, steps, +3 more. Returns: result
screen_recording_statusB
Return active screen recording status for a device.
Inputs: device (required). Returns: result
screenshotA
Take a screenshot of the device screen. Returns a base64-encoded JPEG. Use this to SEE what's on screen before deciding what to tap.
Routes through the shared compressed screenshot path (half-resolution JPEG, cross-platform) instead of a raw full-res PNG: a raw PNG base64 string overflows the MCP tool-result token cap on content-heavy screens, so the client falls back to text/OCR and never sees the pixels. The downscale cuts the payload ~4-8x so most screens stay under the cap. (The lasting fix is returning an MCP image-content block; tracked as feature #8.)
Inputs: device (required). Returns: result
screenshot_annotatedA
Take a screenshot with numbered element labels overlaid on interactive elements. The numbers correspond to element indices from get_elements(). Use this when you want to SEE the screen with elements visually labelled. Returns base64-encoded PNG.
Inputs: device (required). Returns: result
screenshot_croppedA
Take a screenshot of a specific region of the screen. Coordinates are in device pixels. Use this to zoom in on a specific area (e.g., a form field, a notification, a chart). Returns base64-encoded JPEG.
Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +1 more. Returns: result
search_appsA
Search installed apps by name. Case-insensitive. Returns Android packages or iOS bundle ids. Example: search_apps('tiktok') → [{"name": "TikTok", "package": "com.zhiliaoapp.musically"}]
Inputs: query (required), device (required). Returns: result
speak_textA
Make the phone speak text aloud using its built-in TTS engine.
Works whether the agent runs on the phone or on a PC — the call always goes through the Ghost portal app running on the device.
Args: device: ADB serial. This tool is Android-only. text: Text to speak. rate: Speech rate multiplier (0.5 = slow, 1.0 = normal, 1.5 = fast).
Inputs: text (required), device (required), rate. Returns: result
start_screen_recordingB
Start recording the device screen.
iOS uses WDA MJPEG captured through ffmpeg. Android uses adb screenrecord.
Inputs: device (required), filename. Returns: result
stop_screen_recordingC
Stop a running device screen recording and save the MP4.
Inputs: device (required). Returns: result
swipeA
Swipe from (x1,y1) to (x2,y2). Use for scrolling, pulling down notifications, etc. Common patterns: scroll down = swipe(dev, 540, 1400, 540, 600) scroll up = swipe(dev, 540, 600, 540, 1400)
Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +2 more. Returns: result
tapB
Tap at exact pixel coordinates (x, y) on the device screen.
Inputs: x (required), y (required), device (required). Returns: result
tap_elementA
Tap a UI element by its index from get_elements(). Call get_elements() first to see what's on screen and get element indices.
Inputs: idx (required), device (required). Returns: result
toggle_overlayA
Toggle the numbered element overlay on the device screen. When on, interactive elements get visible numbered labels that match get_elements() indices. Useful for visual debugging or when sending screenshots to a vision model.
Inputs: device (required), visible. Returns: result
type_textA
Type text into the currently focused input field. Tap an input field first to focus it. Spaces are supported. Non-ASCII input is transliterated to the closest ASCII (adb input text is ASCII-only); for full-fidelity emoji/CJK use type_unicode() instead.
Inputs: text (required), device (required). Returns: result
type_unicodeA
Type unicode text into the focused field. Android uses ADBKeyboard when configured; iOS uses WDA text entry. Use type_text() for plain ASCII.
Inputs: text (required), device (required). Returns: result
waitB
Pause execution for a fixed number of seconds.
Inputs: seconds. Returns: result
wait_for_textC
Wait until text appears on screen and return visible text context.
Inputs: text (required), device (required), timeout. Returns: result
web_searchA
Open a web search in whatever browser is on the device.
Faster than: launch Chrome → tap address bar → type → submit. Useful when the user asks "search for X" or you need to look up info that's not on the current screen. Picks the first installed browser from a fallback chain (Chrome → Firefox → Samsung Internet → Edge → Brave → Opera → Vivaldi → DuckDuckGo Browser → system default), so it works even if Chrome is missing.
Args: device: ADB serial or ios:. query: Free-text search terms (don't pre-escape — handled here). engine: "google" (default), "ddg" / "duckduckgo", "bing", or "brave". bundle_id: Optional iOS browser bundle id override, e.g. com.google.chrome.ios.
Inputs: query (required), device (required), engine, bundle_id. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 66 tools
Many tools have overlapping purposes (e.g., tap_element vs tap, get_elements vs get_screen_tree vs get_screen_xml, screenshot variants, OCR variants, app listing tools, and CRM lookup tools). Even with descriptions, an agent may struggle to select the correct tool.
Mostly snake_case verb_noun, but there are bare verbs (swipe, tap, wait), noun-only names (device_health, app_state, screen_recording_status), and inconsistent prefix usage (crm_lookup_contact vs lookup_lead). The pattern is readable but not fully consistent.
66 tools is excessive for a single server, far beyond the typical 3-15 well-scoped set. This causes usability issues despite the broad domain.
The server covers many mobile automation areas (UI, OCR, workflows, skills, crashes, CRM), but notable gaps remain—no direct app uninstall, no lead update/reply, no device setting controls. Agents can work around some gaps via workflows.
Maintenance
Related MCP Connectors
AI agents on real phones: run tasks, read the screen, tap, type and install apps.
Cloud Android phones for AI agents: create a phone, read the screen, tap, type, install APKs, park.
Give your AI agent a real cloud Android phone: push APKs, drive it, get screenshots and logs.
Let your AI agent control real iPhones: screenshots, tap, swipe, type and open apps.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceAn MCP server that lets AI agents control iOS and Android devices (tap, scroll, type, take screenshots, read UI trees, and run code). Works with multiple devices at the same time.91 npm53MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to control Android devices via ADB, providing tools for screen capture, tap/swipe/input, app management, and diagnostics. Works locally over USB or WiFi and integrates with MCP clients like Claude and Cursor.1MIT
- AlicenseNot gradedqualityBmaintenanceControls Android and iOS devices from AI agents through a token-efficient MCP server, enabling observation, tapping, typing, gestures, navigation, assertions, and safety-gated actions.2MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to control a physical Android device or Docker-hosted emulator over ADB by taking screenshots and performing taps, swipes, key presses, text input, shell commands, app launches, and device info queries. It runs as a local MCP stdio server alongside a web UI and REST API for remote access.1MIT