Fail unless app_id is in the foreground.

Fail unless text becomes visible on screen within timeout seconds.

Backend, platform and the actions this device supports.

Clear the focused text field.

(Re)connect adb and the Portal for a device; use after the network path came back.

Package / bundle id of the foreground app.

Disconnect a TCP/IP adb device (adb disconnect host:port) and drop its session.

Dismiss one notification (by key, or package/title) or every clearable one.

Health check of adb, the Portal and the device (mobilerun doctor).

Double-tap at (x, y) or a mark.

Returns text verbatim: a check that the MCP transport is alive (no device access).

Mark the end of the task (the server keeps listening; the next call starts fresh). reason: short summary of what was done. outcome: success (goal state verified) | partial | failure. goal_type: play_media | send_message | send_email | purchase | post | open_app | search | navigate | other. For send_message, send_email, purchase and post, success is refused unless you looked at the screen (read_screen / perceive_screen) after your last action. Success is also refused while fewer findings than the plan's target_count are recorded. failure is never refused.

Run JavaScript in the foreground browser page and return its JSON result.

Search the device's media index by name, newest first: images, videos, audio and documents (downloads included). kind: image | video | audio | document | any. Returns content:// URIs for open_file (limit default 10, cap 25). With path, searches that shared-storage folder by file name instead (any file type).

Nodes matching every given filter (exact text/desc/resource_id/class_name, or *_contains substrings), including off-screen ones.

Like find_nodes, limited to nodes inside the visible screen.

The clipboard's text (Android needs the Mobilerun Keyboard as the active IME).

Battery, screen power, foreground app, size, storage, network addresses, volume.

Active media sessions (app, playback state, title/artist) and the music volume.

Plain screenshot as an image.

Compact accessibility tree (class, id, label, flags C/L/E/S/K/P, bounds).

How to use this server well. Topics: overview, shortcuts, text_entry, failures, ledger, safety, stop, efficiency, full.

Grant a runtime permission (android.permission.*) to an app.

Install an APK (host path) on the device.

Press a key by mobilerun-core name (back, home, menu, enter, delete, escape, tab, space, search, page_up, page_down, volume_up, volume_down, wakeup, media_play_pause, ...) or by Android keycode number.

Open an app by name (fuzzy) or exact package_name. An ambiguous name returns ranked candidates instead of guessing. If the app is already in the foreground it is left as is (already_foreground=true) unless force=true.

Deep links into an app, best first. source: shortcut (launcher shortcut, app-shortcut://pkg/id, open with open_deeplink) > resolved (curated and verified on this device) > catalog (curated) > discovered (declared by the app's intent filters).

List installed apps (user apps only unless include_system_apps=true). include_protected_apps is honoured on Mobilerun Cloud devices only (as in mobilerun-core).

Devices you can control. scope: local (adb devices) | cloud (Mobilerun Cloud, needs MOBILERUN_CLOUD_API_KEY) | all. Cloud filters: state (creating, assigned, ready, terminated, ...), type, name, country, page, pageSize (or a filters dict). Any listed id works as the device argument of every tool.

Press and hold at (x, y), a mark (som_id) or a get_state element (index). Hold time: duration_ms or ms (default 1000).

Long press at (x, y) (mobilerun agent action).

Search installed apps by name or package; returns ranked candidates with scores.

Update a plan step: pending | in_progress | done | skipped | failed. Put facts you read off the screen in note; the pixels are gone next turn.

Control playback in any app without touching the screen: play, pause, play_pause, next, previous, stop, rewind, fast_forward. Goes to the active media session; package_name (from get_media_sessions) is checked against it and reported.

Toggle mute on the media stream (muted=true/false forces a state). Muting remembers the previous level for unmute.

Tap one of a notification's own buttons (reply, archive, stop...); reply_text fills an inline reply field and sends it. Best-effort: it drives the notification shade.

Start an app and wait until it is in front and idle.

Dispatch a deep link / intent (default action VIEW), optionally pinned to a package.

Jump straight to a screen via a URI, an app-shortcut://pkg/id from list_app_deeplinks, or an intent action. package_name / app_name pin the target app. intent:, file:, content: and javascript: URIs are refused.

Open a file in its default viewer. uri: a content://media/... URI from find_files (never build one by hand); path: a file under shared storage.

Open the recent-apps overview.

LOOK at the screen: an annotated screenshot plus every element and its tap point.

e is an array whose INDEX is the som_id (first entry = som_id 1): [center_x, center_y, name, flags] (name/flags omitted when empty). elements is the same list as text. Box colours: BLUE tappable, GREEN text input (type_text), MAGENTA scrollable, AMBER toggle, GREY nothing declared, RED on-host vision (detector/OCR; a good guess, not a fact). Flags (only when true): e editable, c checked, o unchecked, d disabled, f focused, l long-pressable, ? low-confidence vision box, w scroll host (aim inside it). offscreen: text that exists but is not on screen (cannot be tapped). detail="full" adds the visual pass (OmniParser YOLOv8 icon detector + OCR) for icons the tree does not describe; perception_tier reports tree_only or full. description is logged only. ids go stale after any action.

Is the device reachable? For adb devices, reports the Portal transport (http or content_provider).

Press home, back or enter (kept for old clients; prefer press_home/back/enter).

Press Back (also closes the keyboard without leaving the screen).

Press Enter (submits search bars and forms).

Press the Home button.

Current status-bar notifications, newest first, without touching the screen: key, app, title, text, action labels. Ongoing ones (music, navigation, downloads) only with include_ongoing=true; package_name filters to one app; limit default 20, cap 30. With the safety policy on, banking and authenticator notifications are withheld.

Read the screen now (waits for it to stop moving first): the screen drawn as a character grid, each element a box with its som_id and label, then a table of what can be acted on: som (tap by this), in (som_id of the smallest box containing it), flg, label (only when it did not fit on the grid). Flags: * tappable, e text input (type_text, not tap), S scrollable, c toggle ON, o toggle OFF, l long-pressable, d disabled, - nothing declared (usually still tappable). N+k = som_id N plus k more elements with exactly those bounds; tap N. Header IDLE/BUSY: BUSY means it was still moving when the wait expired. Use perceive_screen instead for how something looks, when this ends with ESCALATE, or when what you need is missing.

Record one item you found. quote must be copied exactly from the CURRENT screen.

Compatibility no-op: screenshots use the Portal / adb screencap, no prompt is needed.

Find contacts by (partial) name and return their phone numbers.

Which app would open this URI (or intent action such as android.settings.WIFI_SETTINGS)? handler_kind: app | browser_only | none.

Plain screenshot. hide_overlay hides the Portal's element overlay first.

Take a screenshot, save it as a PNG file and return the path.

[width, height] in pixels.

Scroll the content in direction (up | down | left | right) by distance (fraction of the screen). verify=true reports whether the screen actually moved (mobilerun-core).

Scroll the content down (reveal what is below): a centered swipe over half the screen (amount), or inside a scrollable mark (som_id).

Scroll the content left (reveal what is to the left).

Scroll the content right (reveal what is to the right).

Two modes. With x1, y1, x2, y2: drag the content from one point to the other (precise scroll). With text: scroll in direction until an element containing text is visible and return its mark.

Scroll until a matching node is on screen; result is the node (or null).

Scroll the content up (reveal what is above).

Put text on the clipboard.

Start a plan checklist. target_count > 0 means 'N items must be recorded' before end_session(success) is allowed. With 3+ steps and a search_query, the first web search rides along in the reply.

Install and enable the Mobilerun Portal on the device (mobilerun setup); path installs a specific Portal APK.

Start an app by id (Android package / iOS bundle id), optionally a specific activity.

Force-stop an app; clear_data also wipes its data.

Whether this device supports a Device action (e.g. execute_script, get_clipboard).

Swipe from (x1, y1) to (x2, y2) over duration_ms / ms (default 300). mobilerun agent form: coordinate=[x, y], coordinate2=[x, y], duration in seconds.

One-call Android actions (action = the verb; verb= is accepted too). Verbs: set_alarm(hour, minute, label), set_timer(seconds, label), dial(phone_number), compose_sms(phone_number, body), add_calendar_event(title, start, end, location, notes; ISO datetimes), share_text(text, subject), navigate( destination, mode drive|walk|bike|transit). dial/compose_sms only prefill; the user sends.

Tap at (x, y) or at the center of a numbered mark (som_id from perceive_screen / read_screen). stealth=true uses mobilerun-core's humanized tap.

Tap a text (or node) and wait until the UI has been idle for idle seconds.

Tap the center of a node returned by find_nodes / find_nodes_on_screen.

Tap the first on-screen node whose text/description contains text.

The device clock.

Type text (mobilerun). index taps that get_state element first; stealth=true types key by key like a person (mobilerun-core), at wpm words per minute.

Type into the focused field (tap it first, or pass som_id). submit presses Enter after.

Raw UI snapshot (a11y_tree, phone_state, device_context, ...), as Device.ui().

The UI snapshot serialized as JSON text.

UI snapshot that retries past a dead or empty accessibility tree.

Uninstall an app.

Pre-check a planned action against the safety policy (and, for our action set, that its target exists) without doing it. gesture_type is the action (tap, type_text, launch_app, open_deeplink, ...); target is the app name/package, deep-link URI or text you plan to use. Returns allowed=false with a category for blocked apps and text.

Check an outcome against the live screen. kind: text (visible), gone (not visible), app (foreground package or name), activity, changed (the last action changed the screen).

Lower the music volume by steps.

Raise the music volume by steps.

LONG waits only (downloads, uploads, processing, status changes); gestures already settle. With text / package / activity: wait until that is on screen (gone=true: until it disappears). Otherwise wait until loading finishes (no progress bar or "Loading" text) and the screen is still; condition is echoed back, you judge the returned state. Timeouts: timeout_ms (default 5000, max 30000) / poll_interval_ms (default 500, min 100), or timeout / interval in seconds.

Wait until app_id is in the foreground.

Wait until the UI stops changing.

Poll find_nodes until something matches (or timeout, returning []).

Wait until the UI differs from now.

Wait until a node containing text exists (off-screen nodes count).

Collect device events for up to timeout_seconds (default 10, max 30), returning early once max_events (default 50) arrive: foreground app, keyboard, screen content, notifications posted/removed. kinds filters: foreground, keyboard, screen, notifications.

Search the web for how to do something in an app ('how to in android'). Returns ranked sources (title, url, snippet) and, with TAVILY_API_KEY set, a synthesized answer. max_results 1-10 (default 5); topic general | news. The screen overrules results.