# mobilerun-mcp

**An MCP server that controls phones: Android (physical, emulator, redroid; ARM or x86_64) over
adb, iOS through ios-portal, and Mobilerun Cloud devices.** Everything runs on the host, so nothing
ARM-only has to run on the device.

94 core tools covering droidrun's [mobilerun-core](https://pypi.org/project/mobilerun-core/) Device API,
screen perception, gestures, apps, intents, and device control. Mobilerun agent actions and
Mobilerun Cloud platform tools are detached into dedicated servers.

### Architecture & Servers

- **`mobilerun-mcp` (Core Server, default)**: 94 tools focusing on `mobilerun-core` Device API,
  screen perception (`read_screen`, `perceive_screen`), input gestures, system intents, and app control.
- **`mobilerun-agent-mcp` (Agent Server)**: Detached server for `mobilerun` agent actions
  (`get_state`, `click`, `type_secret`), background agent tasks, and macro replay.
- **`mobilerun-cloud-mcp` (Cloud Server)**: Detached server for Mobilerun Cloud platform management
  (cloud devices, credentials, workflows).

```
  MCP client (Claude Code, Cursor, ...)
        |  stdio, or HTTP (--http)
  mobilerun-mcp ---- adb ------------------> Android device (Mobilerun Portal)
     |   '---------- mobilerun-core --------> iOS (ios-portal), Portal-HTTP-only Android,
     |                                        Mobilerun Cloud
     '-- host-side OCR (tesseract) and icon detector (OmniParser v2, onnxruntime)
```

**Jump to:** [Quick start](#quick-start) · [Troubleshooting](#troubleshooting) · [Calling the tools](#calling-the-tools) · [Tool reference](#tool-reference) · [Configuration](#configuration)

## Why this exists

Android MCP servers that run *on the phone* are convenient, but they depend on on-device native
libraries built for ARM. On an x86_64 device (redroid, an emulator, an Android-x86 box) those
libraries run under a translation layer that cannot execute some instructions, and the app
crashes. This server keeps all the heavy lifting on the host: it reads the screen through the
[Mobilerun Portal](https://github.com/droidrun/mobilerun-portal) accessibility service and drives
gestures through the Portal and `adb`. Nothing ARM-only runs on the device.

It also works over any network `adb` works over (LAN, VPN, `adb connect`), not only the same Wi-Fi.

## Quick start

### Requirements

| Package | Needed for | Install |
|---|---|---|
| `adb` (platform-tools) | device access | [download](https://developer.android.com/tools/releases/platform-tools), `apt install adb`, `brew install android-platform-tools` |
| `uv` (or Python 3.11+) | environment and dependencies | [install](https://docs.astral.sh/uv/getting-started/installation/) |
| `mobilerun` CLI | installing the Portal; `run_task` | `uv tool install mobilerun` |
| `tesseract` (optional) | OCR on screens with a sparse accessibility tree | [install](https://tesseract-ocr.github.io/tessdoc/Installation.html), `apt install tesseract-ocr`, `brew install tesseract` |

Python dependencies (`fastmcp`, `mobilerun-core[local]`, `onnxruntime`, `numpy`, `pillow`, ...) are
installed with the project. The icon detector for `perceive_screen(detail="full")` (OmniParser v2,
~80 MB, AGPL-3.0) downloads on first use to `~/.cache/mobilerun-mcp`.

You also need an Android device that `adb` can reach: a USB phone
([enable USB debugging](https://developer.android.com/studio/debug/dev-options)), an x86_64
[emulator](https://developer.android.com/studio/run/managing-avds), or
[redroid](https://github.com/remote-android/redroid-doc):

```bash
docker run -itd --privileged -p 5555:5555 redroid/redroid:12.0.0-latest
adb connect localhost:5555
```

### Install

```bash
adb devices                                   # note the serial

# Mobilerun Portal: the accessibility service the server reads the screen through
mobilerun setup -d <serial>
mobilerun ping -d <serial>                    # Portal is installed and accessible

# Server
git clone https://github.com/Hi-im-Connect/mobilerun-mcp.git && cd mobilerun-mcp
uv venv --python 3.13 .venv && uv pip install --python .venv/bin/python -e .
```

On Windows use `.venv\Scripts\python.exe` in place of `.venv/bin/python`.

<details>
<summary>Manual Portal install</summary>

Download the APK from [Portal releases](https://github.com/droidrun/mobilerun-portal/releases), then:

```bash
adb -s <serial> install -r <portal.apk>
# enable it: Settings > Accessibility > Mobilerun Portal, or headless (replaces other enabled services):
adb -s <serial> shell settings put secure enabled_accessibility_services com.mobilerun.portal/com.mobilerun.portal.service.MobilerunAccessibilityService
adb -s <serial> shell settings put secure accessibility_enabled 1
```

</details>

### Register with your MCP client

**Claude Code**

```bash
claude mcp add --scope user mobilerun -e MOBILERUN_DEVICE=<serial> -- "$PWD/.venv/bin/python" -m mobilerun_mcp
```

**Claude Desktop, Cursor, others:** add to the client's MCP config and restart it.

```json
{
  "mcpServers": {
    "mobilerun": {
      "command": "/path/to/mobilerun-mcp/.venv/bin/python",
      "args": ["-m", "mobilerun_mcp"],
      "env": { "MOBILERUN_DEVICE": "<serial>" }
    }
  }
}
```

| Client | Config file |
|---|---|
| Claude Desktop (macOS) | `~/Library/Application Support/Claude/claude_desktop_config.json` |
| Claude Desktop (Windows) | `%APPDATA%\Claude\claude_desktop_config.json` |
| Cursor | `~/.cursor/mcp.json` |

Client docs: [Claude Code](https://docs.claude.com/en/docs/claude-code/mcp),
[Claude Desktop](https://modelcontextprotocol.io/quickstart/user),
[Cursor](https://docs.cursor.com/context/model-context-protocol).

`MOBILERUN_DEVICE` is optional when exactly one device is attached to `adb`. Every device tool also
takes a `device` argument, so one server can drive several devices:

| `device` / `MOBILERUN_DEVICE` | Target |
|---|---|
| `emulator-5554`, `192.168.1.20:5555`, a USB serial | Android over adb (all tools) |
| `http://host:8080` or `android-http:<url>` | Android through the Portal HTTP API only (`MOBILERUN_ANDROID_PORTAL_TOKEN`) |
| `ios` or `ios:<url>` | iOS via [ios-portal](https://github.com/droidrun/ios-portal) (default `http://127.0.0.1:6643`) |
| `cloud:<id>` or a device UUID | Mobilerun Cloud device (`MOBILERUN_CLOUD_API_KEY`) |

adb-only tools (dumpsys-based ones, shortcuts, files by path) return `[unsupported]` on the others.

HTTP instead of stdio: `.venv/bin/python -m mobilerun_mcp --http` serves
`http://127.0.0.1:4816/mcp`.

### Usage

Ask the agent in natural language ("open Settings and read the Android version"), or call tools
directly. See [Calling the tools](#calling-the-tools) and the [Tool reference](#tool-reference).

## Troubleshooting

| Symptom | Fix |
|---|---|
| `[device_unreachable] no device selected` | `adb devices`, then set `MOBILERUN_DEVICE=<serial>` |
| `adb devices` shows `unauthorized` | Accept the USB debugging prompt on the device; if it does not appear, `adb kill-server` and reconnect |
| `adb devices` empty or `offline` | Replug USB; for a container or remote device run `adb connect <host>:<port>` |
| `adb: command not found` | Install platform-tools, or set `MOBILERUN_ADB_BIN` |
| `Mobilerun Portal is not enabled as an accessibility service` | Enable it under Settings > Accessibility, or rerun `mobilerun setup` |
| `mobilerun setup` stalls after "Found Portal APK" | Play Protect is scanning the install (redroid, emulators with Google Play): `adb -s <serial> shell settings put global verifier_verify_adb_installs 0` and `... package_verifier_enable 0`, then rerun |
| Client lists no `mobilerun` tools | Restart the client and check the config path. Run `.venv/bin/python -m mobilerun_mcp` by hand: a FastMCP banner followed by waiting is healthy, a traceback is the cause |
| `[stale_som_id]` | An action changed the screen; call `perceive_screen` again |
| Slow first launch | Cold starts take 20 to 30 s on slow devices; `launch_app` waits for the app |
| `web_search` error | DuckDuckGo throttled the request; retry or set `BRAVE_API_KEY` |
| Sparse text on image-heavy screens | Install `tesseract` for OCR, or tap by coordinates from the screenshot |

Issues: [github.com/Hi-im-Connect/mobilerun-mcp/issues](https://github.com/Hi-im-Connect/mobilerun-mcp/issues) (include the error and `adb devices` output).

## How it works

The agent works in a **perceive, act, verify** loop:

1. `perceive_screen` returns a numbered list of everything tappable or readable (`som_id`s) and an
   annotated screenshot with the same numbers drawn on it.
2. An action tool (`tap`, `type_text`, `launch_app`, ...) waits for the screen to settle, then
   returns a `post_action_observation`: foreground app, element count, keyboard state, the top
   labels on screen and whether the screen changed. That block is the verification step.
3. `som_id`s describe one captured screen. After any action they are stale and the server refuses
   them (`stale_som_id`), so the agent can never tap something that has moved.

Details that matter in practice:

- **Elements without text.** Icon-only buttons are numbered too, as long as the app exposes them
  to Android's accessibility service, which is the case for standard apps.
- **Cold starts.** Launching an app waits for that app to reach the foreground. On a slow device a
  cold start can take 20 seconds; an app that is already on top returns immediately.
- **Gestures** go through the Portal's accessibility gestures (fast, and accepted by system UI
  such as the notification shade), with `adb input` as the fallback.
- **Errors** look like `[code] message (hint: ...)`: `device_unreachable`, `policy_blocked`,
  `stale_som_id`, `unknown_som_id`, `element_not_found`, `app_not_found`, `timeout`,
  `unsupported`, `invalid_argument`, `not_permitted`, `plan_incomplete`.

## Calling the tools

Every capability is an MCP tool: a name and a JSON `arguments` object. The client makes the call
when you describe what you want, or you can invoke a tool by name. On the wire:

```json
{"method": "tools/call", "params": {"name": "tap", "arguments": {"som_id": 2}}}
```

For every tool:

- **Optional arguments** can be left out; `null` also means "not given".
- **`device`** is an optional argument on every tool that touches a device (an adb serial). Leave it
  out to use `MOBILERUN_DEVICE`, or the only attached device.
- **State-changing tools** (`tap`, `swipe`, `type_text`, `launch_app`, ...) wait for the screen to
  settle and return `{"ok": true, "action": ..., "post_action_observation": {...}}`.
- **Read tools** return a JSON object; `perceive_screen` and the screenshot tools also return an image.
- **Errors** come back as `[code] message (hint: ...)`, for example `[stale_som_id] ...`.

The `post_action_observation` block tells the agent what the screen looks like after the action:

| Field | Meaning |
|---|---|
| `foreground_app`, `package`, `activity` | What is in front now |
| `element_count` | How many numbered elements are on screen |
| `keyboard_visible` | Whether the on-screen keyboard is up |
| `top_labels` | The first few labels on screen, in reading order |
| `screen_changed` | Whether the screen differs from before the action |
| `loading_indicator_present` | A spinner or progress bar is visible |
| `settled`, `settle_ms` | The screen stopped changing before the wait ended; how long that took |
| `screen_changed_confidence` | `low` when the screen never settled |
| `seen_before` | This screen was already seen N actions ago (going in circles?) |
| `sensitive_foreground` | A banking / payment / authenticator app is in front |
| `hint` | What to do next |

### A typical session

Search Contacts for "ali". Calls are written `tool arguments`; outputs are real.

**1. Look at the screen.**

```
perceive_screen {}
```

The reply is a JSON object with `foreground_app`, `package`, `activity`, `keyboard_visible`,
`screen_size`, `perception_tier`, `e` (`[x, y, name, flags]` per `som_id`),
`mark_count`, `ocr_used` and `elements`. Here `e` is
`[[56, 104, "Open navigation drawer"], [664, 104, "Search contacts"], [224, 104, "Contacts"], [360, 230, "A / Ali Omar", "l"], [632, 1096, "Create new contact"]]`
and `elements` contains:

```
  1 [button] "Open navigation drawer" @(56,104)
  2 [button] "Search contacts" @(664,104)
  3 [text] "Contacts" @(224,104)
  4 [button] "A / Ali Omar" @(360,230)
  5 [button] "Create new contact" @(632,1096)
```

The leading number is the `som_id`; `@(x,y)` is the tap point. An annotated screenshot with the same
numbers is returned alongside the JSON.

**2. Tap the search icon by its number.**

```
tap {"som_id": 2}
```

The reply has `"ok": true` and `keyboard_visible: true` in `post_action_observation`. Any action
invalidates the numbers: reusing `som_id` 2 now fails with `stale_som_id` until `perceive_screen` is
called again.

**3. Type.**

```
type_text {"text": "ali"}
```

```json
{
  "ok": true,
  "action": "type_text",
  "chars": 3,
  "post_action_observation": {
    "foreground_app": "Contacts",
    "package": "com.android.contacts",
    "activity": "PeopleActivity",
    "element_count": 5,
    "keyboard_visible": true,
    "top_labels": [
      "stop searching",
      "ali",
      "Clear search",
      "Ali Omar"
    ],
    "screen_changed": true,
    "loading_indicator_present": false,
    "settled": true,
    "settle_ms": 922,
    "sensitive_foreground": false,
    "hint": "Settled after the action. Judge the result from this observation ..."
  }
}
```

**4. Verify.**

```
verify_action {"expected": "Ali Omar"}
```

```json
{
  "expected": "Ali Omar",
  "state": {"foreground_app": "Contacts", "element_count": 4, "keyboard_visible": true,
            "top_labels": ["stop searching", "ali", "Clear search", "Ali Omar"], "...": "..."},
  "verified": true,
  "evidence": "text=\"Ali Omar\"",
  "foreground": "com.android.contacts",
  "visible_text": ""
}
```

## Tool reference

94 tools, plus `adb` when `MOBILERUN_MCP_ENABLE_ADB=1`. Every tool that acts on a device takes an optional `device` (adb serial, `ios`, `cloud:<id>`, or a Portal URL). `[name=default]` is optional. Generated by `scripts/gen_reference.py`.

### Perception

See the screen. `read_screen` (text grid) or `perceive_screen` (annotated image) first; act by `som_id`.

| Tool | What it does | Arguments |
|---|---|---|
| `perceive_screen` | LOOK at the screen: an annotated screenshot plus every element and its tap point. | [description] [detail] [include_image=true] [ocr=auto] [max_marks=150] [lang=eng] |
| `read_screen` | Read the screen now (waits for it to stop moving first): the screen drawn as a character grid, each element a box with its som_id and label, then a table of what can be acted on: som (tap by this), in (som_id of the smal | none |
| `get_ui_tree` | Compact accessibility tree (class, id, label, flags C/L/E/S/K/P, bounds). | [max_depth=8] |
| `get_screenshot` | Plain screenshot as an image. | none |
| `screenshot` | Plain screenshot. | [hide_overlay=false] |
| `screenshot_path` | Take a screenshot, save it as a PNG file and return the path. | none |

```
perceive_screen {}
perceive_screen {"description": "search bar", "detail": "full"}
read_screen {}
get_ui_tree {}
get_screenshot {}
screenshot {}
screenshot_path {}
```

### Gestures, typing and keys

Every action settles the screen and returns `post_action_observation`. Target with `x`/`y`, a `som_id`, or (mobilerun style) an `index` from `get_state`.

| Tool | What it does | Arguments |
|---|---|---|
| `tap` | Tap at (x, y) or at the center of a numbered mark (som_id from perceive_screen / read_screen). | [x] [y] [som_id] [stealth=false] |
| `double_tap` | Double-tap at (x, y) or a mark. | [x] [y] [som_id] |
| `long_press` | Press and hold at (x, y), a mark (som_id) or a get_state element (index). | [x] [y] [som_id] [index] [duration_ms] [ms] |
| `long_press_at` | Long press at (x, y) (mobilerun agent action). | x y |
| `swipe` | Swipe from (x1, y1) to (x2, y2) over duration_ms / ms (default 300). | [x1] [y1] [x2] [y2] [duration_ms] [ms] [coordinate] [coordinate2] [duration] |
| `scroll_down` | Scroll the content down (reveal what is below): a centered swipe over half the screen (amount), or inside a scrollable mark (som_id). | [amount=0.5] [som_id] |
| `scroll_up` | Scroll the content up (reveal what is above). | [amount=0.5] [som_id] |
| `scroll_left` | Scroll the content left (reveal what is to the left). | [amount=0.5] [som_id] |
| `scroll_right` | Scroll the content right (reveal what is to the right). | [amount=0.5] [som_id] |
| `scroll` | Scroll the content in direction (up / down / left / right) by distance (fraction of the screen). | direction [distance=0.5] [ms=300] [verify=false] |
| `scroll_to` | Two modes. | [x1] [y1] [x2] [y2] [duration_ms=300] [text] [direction=down] [max_scrolls=8] |
| `type_text` | Type into the focused field (tap it first, or pass som_id). | text [clear=false] [submit=false] [som_id] |
| `type` | Type text (mobilerun). | text [index] [clear=false] [wpm] [stealth=false] |
| `press_home` | Press the Home button. | none |
| `press_back` | Press Back (also closes the keyboard without leaving the screen). | none |
| `press_enter` | Press Enter (submits search bars and forms). | none |
| `open_recent_apps` | Open the recent-apps overview. | none |
| `key` | Press a key by mobilerun-core name (back, home, menu, enter, delete, escape, tab, space, search, page_up, page_down, volume_up, volume_down, wakeup, media_play_pause, ...) or by Android keycode number. | name_or_code |

```
tap {"som_id": 4}
tap {"x": 540, "y": 1200}
double_tap {}
long_press {"som_id": 4}
long_press {"index": 7, "ms": 800}
long_press_at {"x": 1, "y": 1}
swipe {"x1": 360, "y1": 1000, "x2": 360, "y2": 300}
swipe {"coordinate": [360, 1000], "coordinate2": [360, 300], "duration": 0.5}
scroll_down {}
scroll_up {}
scroll_left {}
scroll_right {}
scroll {"direction": "down"}
scroll_to {"text": "Battery"}
scroll_to {"x1": 360, "y1": 900, "x2": 360, "y2": 400}
type_text {"text": "hello", "som_id": 3, "submit": true}
type {"text": "hello", "index": 5, "clear": true}
press_home {}
press_back {}
press_enter {}
open_recent_apps {}
key {"name_or_code": "back"}
```

### Apps and deep links

| Tool | What it does | Arguments |
|---|---|---|
| `launch_app` | Open an app by name (fuzzy) or exact package_name. | [app_name] [package_name] [force=false] [package] |
| `start_app` | Start an app by id (Android package / iOS bundle id), optionally a specific activity. | [app_id] [activity] [package] |
| `lookup_app` | Search installed apps by name or package; returns ranked candidates with scores. | [app_name] [query] [limit=5] |
| `list_apps` | List installed apps (user apps only unless include_system_apps=true). | [include_system_apps=false] [include_protected_apps=false] [system=false] |
| `list_app_deeplinks` | Deep links into an app, best first. | [package_name] [app_name] [package] |
| `resolve_deeplink` | Which app would open this URI (or intent action such as android.settings.WIFI_SETTINGS)? | uri |
| `open_deeplink` | Jump straight to a screen via a URI, an app-shortcut://pkg/id from list_app_deeplinks, or an intent action. | uri [package_name] [app_name] [package] |

```
launch_app {"app_name": "Clock"}
launch_app {"package_name": "com.android.settings", "force": true}
start_app {}
lookup_app {}
list_apps {}
list_app_deeplinks {}
resolve_deeplink {"uri": "https://example.com"}
open_deeplink {"uri": "android.settings.WIFI_SETTINGS"}
open_deeplink {"uri": "app-shortcut://com.android.settings/manifest-shortcut-wifi"}
```

### System intents and contacts

| Tool | What it does | Arguments |
|---|---|---|
| `system_intent` | One-call Android actions (action = the verb; verb= is accepted too). | [action] [verb] [hour] [minute] [seconds] [label] [phone_number] [body] [title] [start] [end] [location] [notes] [text] [subject] [destination] [mode=drive] [skip_ui=true] |
| `resolve_contact` | Find contacts by (partial) name and return their phone numbers. | name [limit=5] |

```
system_intent {"action": "set_alarm", "hour": 7, "minute": 30, "label": "wake"}
system_intent {"action": "navigate", "destination": "Cairo Tower", "mode": "walk"}
resolve_contact {"name": "Ali"}
```

### Notifications

| Tool | What it does | Arguments |
|---|---|---|
| `read_notifications` | Current status-bar notifications, newest first, without touching the screen: key, app, title, text, action labels. | [package_name] [include_ongoing=false] [limit=20] [package] |
| `dismiss_notification` | Dismiss one notification (by key, or package/title) or every clearable one. | [key] [package] [title] [clear_all=false] |
| `notification_action` | Tap one of a notification's own buttons (reply, archive, stop...); reply_text fills an inline reply field and sends it. | action [key] [package] [title] [reply_text] |

```
read_notifications {}
dismiss_notification {}
notification_action {"action": "list"}
```

### Media and volume

| Tool | What it does | Arguments |
|---|---|---|
| `get_media_sessions` | Active media sessions (app, playback state, title/artist) and the music volume. | [include_system=false] |
| `media_control` | Control playback in any app without touching the screen: play, pause, play_pause, next, previous, stop, rewind, fast_forward. | [command] [package_name] [action] |
| `volume_up` | Raise the music volume by ``steps``. | [steps=1] |
| `volume_down` | Lower the music volume by ``steps``. | [steps=1] |
| `mute` | Toggle mute on the media stream (muted=true/false forces a state). | [muted] |

```
get_media_sessions {}
media_control {}
volume_up {}
volume_down {}
mute {}
```

### Files

| Tool | What it does | Arguments |
|---|---|---|
| `find_files` | Search the device's media index by name, newest first: images, videos, audio and documents (downloads included). | [query] [kind=any] [limit=10] [path] [max_depth=6] |
| `open_file` | Open a file in its default viewer. | [uri] [path] |

```
find_files {}
open_file {}
```

### Waiting and checking

| Tool | What it does | Arguments |
|---|---|---|
| `wait_for` | LONG waits only (downloads, uploads, processing, status changes); gestures already settle. | [condition] [timeout_ms] [poll_interval_ms] [text] [package] [activity] [gone=false] [timeout] [interval] |
| `watch_device_events` | Collect device events for up to timeout_seconds (default 10, max 30), returning early once max_events (default 50) arrive: foreground app, keyboard, screen content, notifications posted/removed. | [timeout_seconds] [max_events=50] [duration] [interval=0.5] [kinds] |
| `validate_action` | Pre-check a planned action against the safety policy (and, for our action set, that its target exists) without doing it. | [gesture_type] [target] [action] [x] [y] [som_id] [text] [package] [app_name] [uri] |
| `verify_action` | Check an outcome against the live screen. | expected [kind=text] [timeout=3.0] [use_ocr=false] |

```
wait_for {}
watch_device_events {}
validate_action {}
verify_action {"expected": "Settings is open"}
```

### Plan, findings and research

| Tool | What it does | Arguments |
|---|---|---|
| `web_search` | Search the web for how to do something in an app ('how to <task> in <app> android'). | query [max_results] [topic=general] [limit=5] |
| `set_plan` | Start a plan checklist. | steps [goal] [deliverable] [target_count=0] [search_query] |
| `mark_step` | Update a plan step: pending / in_progress / done / skipped / failed. | index status [note] |
| `record_finding` | Record one item you found. | item quote |
| `end_session` | Mark the end of the task (the server keeps listening; the next call starts fresh). | [reason=agent-end] [outcome=success] [goal_type] [summary] |
| `get_usage_guide` | How to use this server well. | [topic] |

```
web_search {"query": "wifi"}
set_plan {"steps": []}
mark_step {"index": 1, "status": "value"}
record_finding {"item": "Result 1", "quote": "exact text"}
end_session {}
get_usage_guide {}
```

### mobilerun-core Device API

Same names and parameters as `mobilerun_core.Device`. Works on Android (adb or Portal HTTP), iOS and Mobilerun Cloud devices.

| Tool | What it does | Arguments |
|---|---|---|
| `ui` | Raw UI snapshot (a11y_tree, phone_state, device_context, ...), as Device.ui(). | [filter=true] |
| `ui_json` | The UI snapshot serialized as JSON text. | [filter=true] [indent] |
| `ui_with_recovery` | UI snapshot that retries past a dead or empty accessibility tree. | [filter=true] |
| `capabilities` | Backend, platform and the actions this device supports. | none |
| `supports` | Whether this device supports a Device action (e.g. | action |
| `screen_size` | [width, height] in pixels. | none |
| `current_app_id` | Package / bundle id of the foreground app. | none |
| `time` | The device clock. | none |
| `find_nodes` | Nodes matching every given filter (exact text/desc/resource_id/class_name, or *_contains substrings), including off-screen ones. | [text] [desc] [resource_id] [class_name] [text_contains] [desc_contains] [any_contains] [tree] |
| `find_nodes_on_screen` | Like find_nodes, limited to nodes inside the visible screen. | [text] [desc] [resource_id] [class_name] [text_contains] [desc_contains] [any_contains] [tree] |
| `tap_text` | Tap the first on-screen node whose text/description contains text. | text |
| `tap_node` | Tap the center of a node returned by find_nodes / find_nodes_on_screen. | node [stealth=true] |
| `tap_and_wait` | Tap a text (or node) and wait until the UI has been idle for idle seconds. | target [idle=2.0] |
| `scroll_until` | Scroll until a matching node is on screen; result is the node (or null). | [text] [text_contains] [any_contains] [resource_id] [direction=down] [max_swipes=10] [distance=0.35] [settle=0.5] |
| `clear_input` | Clear the focused text field. | none |
| `assert_on` | Fail unless app_id is in the foreground. | app_id |
| `assert_text_visible` | Fail unless text becomes visible on screen within timeout seconds. | text [timeout=5.0] |
| `wait_for_app` | Wait until app_id is in the foreground. | app_id [timeout=10.0] [poll=0.5] |
| `wait_for_idle` | Wait until the UI stops changing. | [timeout=5.0] [poll=0.5] |
| `wait_for_screen_change` | Wait until the UI differs from now. | [timeout=10.0] [poll=0.5] |
| `wait_for_text` | Wait until a node containing text exists (off-screen nodes count). | text [timeout=10.0] [poll=0.5] |
| `wait_for_nodes` | Poll find_nodes until something matches (or timeout, returning []). | [timeout=10.0] [poll=0.5] [text] [desc] [resource_id] [class_name] [text_contains] [desc_contains] [any_contains] [on_screen=false] |
| `open_and_settle` | Start an app and wait until it is in front and idle. | app_id [timeout=15.0] [idle=3.0] |
| `stop_app` | Force-stop an app; clear_data also wipes its data. | app_id [clear_data=false] |
| `install_app` | Install an APK (host path) on the device. | path [replace=false] [grant_permissions=true] |
| `uninstall_app` | Uninstall an app. | app_id |
| `grant_permission` | Grant a runtime permission (android.permission.*) to an app. | package permission |
| `open_deep_link` | Dispatch a deep link / intent (default action VIEW), optionally pinned to a package. | deep_link [package_name] [action] |
| `execute_script` | Run JavaScript in the foreground browser page and return its JSON result. | js |
| `get_clipboard` | The clipboard's text (Android needs the Mobilerun Keyboard as the active IME). | none |
| `set_clipboard` | Put text on the clipboard. | value |

```
ui {}
ui_json {}
ui_with_recovery {}
capabilities {}
supports {"action": "list"}
screen_size {}
current_app_id {}
time {}
find_nodes {"text_contains": "Wi"}
find_nodes_on_screen {}
tap_text {"text": "Settings"}
tap_node {"node": {}}
tap_and_wait {"target": "Settings"}
scroll_until {}
clear_input {}
assert_on {"app_id": "com.android.settings"}
assert_text_visible {"text": "Settings"}
wait_for_app {"app_id": "com.android.settings"}
wait_for_idle {}
wait_for_screen_change {}
wait_for_text {"text": "Settings"}
wait_for_nodes {}
open_and_settle {"app_id": "com.android.settings"}
stop_app {"app_id": "com.android.settings"}
install_app {"path": "/sdcard/Download/a.apk"}
uninstall_app {"app_id": "com.android.settings"}
grant_permission {"package": "com.android.settings", "permission": "android.permission.CAMERA"}
open_deep_link {"deep_link": "https://example.com"}
execute_script {"js": "document.title"}
get_clipboard {}
set_clipboard {"value": "copied text"}
```

### Devices and connection

| Tool | What it does | Arguments |
|---|---|---|
| `get_device_status` | Battery, screen power, foreground app, size, storage, network addresses, volume. | none |
| `list_devices` | Devices you can control. | [scope=local] [state] [type] [name] [country] [page] [pageSize] [filters] |
| `ping_device` | Is the device reachable? | none |
| `connect_device` | (Re)connect adb and the Portal for a device; use after the network path came back. | none |
| `disconnect_device` | Disconnect a TCP/IP adb device (adb disconnect host:port) and drop its session. | none |
| `setup_portal` | Install and enable the Mobilerun Portal on the device (mobilerun setup); path installs a specific Portal APK. | [path] |
| `doctor` | Health check of adb, the Portal and the device (mobilerun doctor). | none |
| `request_screen_capture_permission` | Compatibility no-op: screenshots use the Portal / adb screencap, no prompt is needed. | none |
| `echo` | Returns text verbatim: a check that the MCP transport is alive (no device access). | [text] [message] |

```
get_device_status {}
list_devices {}
ping_device {}
connect_device {}
disconnect_device {}
setup_portal {}
doctor {}
request_screen_capture_permission {}
echo {}
```

### Compatibility

| Tool | What it does | Arguments |
|---|---|---|
| `press` | Press home, back or enter (kept for old clients; prefer press_home/back/enter). | button |

```
press {"button": "back"}
```

### Raw adb

Only when `MOBILERUN_MCP_ENABLE_ADB=1`; refused while a safety policy is on.

| Tool | What it does | Arguments |
|---|---|---|
| `adb` | Run an adb command against the device. | command |

```
adb {"command": "shell dumpsys battery"}
```

## Configuration

| Variable | Default | Meaning |
|---|---|---|
| `MOBILERUN_DEVICE` | the only attached device | Default device (see the `device` table above) |
| `MOBILERUN_MCP_POLICY` | `off` | Safety policy: `off`, `standard`, `strict` |
| `MOBILERUN_MCP_SCOPES` | `read,write` | Set to `read` to expose only read-only tools |
| `MOBILERUN_MCP_ENABLE_ADB` | `0` | Set to `1` to expose the raw `adb` tool |
| `BRAVE_API_KEY` | unset | `web_search` uses Brave when set, DuckDuckGo otherwise |
| `TAVILY_API_KEY` | unset | `web_search` uses Tavily (synthesized answer) when set |
| `MOBILERUN_CLOUD_API_KEY` | unset | Mobilerun Cloud devices, tasks and the cloud platform tools |
| `MOBILERUN_CREDENTIALS` | `config/credentials.yaml` | Secrets file for `type_secret` (mobilerun format) |
| `MOBILERUN_DETECTOR_MODEL` | downloaded | Path to an OmniParser icon-detect `.onnx` |
| `MOBILERUN_IOS_PORTAL_URL`, `MOBILERUN_IOS_PORTAL_TOKEN` | `http://127.0.0.1:6643` | iOS portal for `device="ios"` |
| `MOBILERUN_ANDROID_PORTAL_TOKEN` | unset | Bearer token for Portal-HTTP-only Android targets |
| `MOBILERUN_MCP_HTTP_HOST`, `MOBILERUN_MCP_HTTP_PORT` | `127.0.0.1`, `4816` | Address for `--http` |
| `MOBILERUN_ADB_BIN`, `MOBILERUN_BIN` | on `PATH` | Binary overrides |

### Safety policy

The policy is **off by default**, so an agent can sign in to accounts and use any app.

- `standard` blocks banking, payment and wallet apps, authenticator apps and password managers,
  Luhn-valid card numbers, and fields asking for a card security code.
- `strict` additionally refuses password and PIN fields and national-id numbers.

Blocked actions fail with `[policy_blocked]`. `run_task` and the raw `adb` tool are disabled while
a policy is on, because they cannot be policed. Read `mobilerun://policy` for the active rules.

## Limitations

- **Tested live on redroid 12 (Android 12, x86_64).** Physical phones, other Android versions, iOS
  and cloud devices go through the same code paths but were not driven live here; the cloud tools
  are verified against mocked API responses.
- **Icon detection** (`detail="full"`) is a host-side guess: red boxes, not facts.
- **`media_control` with `package_name`** goes to the active media session (adb cannot address one
  app's session); the reply says when that is a different app.
- **Notification actions and dismissal** drive the notification shade, because `adb` cannot fire a
  PendingIntent. Ongoing notifications cannot be dismissed.
- **Local `run_task`** runs the mobilerun CLI agent; `outputSchema`, `apps`, `credentials`, `files`
  and `stealth` apply to cloud tasks only. The agent's self-report can be wrong: verify on screen.
- **Launcher shortcuts** come from `dumpsys shortcut`; Android elides the path of https shortcut
  URIs there, so those open the app without the exact page.
- Volume commands succeed on redroid but have no audible effect.

## Development

```bash
uv pip install --python .venv/bin/python -e ".[dev]"
.venv/bin/python -m pytest                          # unit tests, no device needed
MOBILERUN_DEVICE=<serial> .venv/bin/python -m pytest -m live     # drives a real device
.venv/bin/ruff check src tests && .venv/bin/ruff format --check src tests
```

- Unit tests run against real output captured from a device (`tests/fixtures`; regenerate with
  `scripts/capture_fixtures.py`). The JavaScript snippets are syntax-checked with `node` when it
  is installed.
- Live tests drive the device through an in-process MCP client and assert observable effects
  (foreground app, screen contents, notification state, page state), not just that a call returned.
  They post notifications, change the media volume and open apps, so use a scratch device.

```
src/mobilerun_mcp/
  adb.py portal.py          transports: adb wrapper, Portal HTTP client
  session.py observe.py     per-device state, settle-and-observe
  models.py marks.py        screen model, numbered marks, signatures
  parsers/                  pure parsers for accessibility state, dumpsys, intent filters, ...
  policy.py ledger.py       safety rules, plan ledger
  tools/                    one small module per tool group
```

## Relationship to other projects

Independent; not affiliated with Mobilerun/droidrun or redroid.

- [mobilerun-core](https://pypi.org/project/mobilerun-core/) (Apache-2.0) is a dependency; its
  `Device` runs on this server's fast Android transport.
- [mobilerun](https://github.com/droidrun/mobilerun) (MIT): the agent's element indexing is adapted
  in `src/mobilerun_mcp/agentui.py`.
- [droidrun/mobilerun-mcp](https://github.com/droidrun/mobilerun-mcp) (Apache-2.0): the cloud tools
  in `src/mobilerun_mcp/tools/cloud.py` are a port of its tool layer.
- [Mobilerun Portal](https://github.com/droidrun/mobilerun-portal) provides screen access on Android.
- [OmniParser v2](https://huggingface.co/microsoft/OmniParser-v2.0) icon detector (AGPL-3.0),
  downloaded at runtime, not redistributed.

## License

MIT. See [LICENSE](LICENSE) and [NOTICE](NOTICE).