Purpose: Launch a mobile app on Android or iOS.

Inputs:

• platform

• appId

• deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { appId }

• success = true when launch was dispatched successfully

• failure_code/retryable when launch dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Define the expected landing screen when it is known

2. Call start_app

3. If needed, wait for transition using wait_for_*

4. Verify with expect_screen

5. If verification fails, retry once or capture a snapshot

Verification Guidance:

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• expect_screen is the authoritative verification step when the landing screen is known

• Do not treat timing or screen change alone as final verification

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → inspect snapshot/logs before retrying

Terminate a mobile app on Android or iOS simulator

Purpose: Restart a mobile app on Android or iOS.

Inputs:

• platform

• appId

• deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { appId }

• success = true when the restart command completed

• failure_code/retryable when restart dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Define the expected landing screen when it is known

2. Call restart_app

3. If needed, wait for transition using wait_for_*

4. Verify with expect_screen

5. If verification fails, retry once or capture a snapshot

Verification Guidance:

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• expect_screen is the authoritative verification step when the reopened screen is known

• Do not treat timing or screen change alone as final verification

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → inspect snapshot/logs before retrying

Reset app data (clear storage) for a mobile app on Android or iOS simulator

Install an app on Android or iOS. Accepts a built binary (apk/.ipa/.app) or a project directory to build then install. platform and projectType are required.

Build a project for Android or iOS and return the built artifact path. Does not install. platform and projectType are required.

Get recent logs from Android or iOS simulator. Returns device metadata and structured logs suitable for AI consumption.

List connected devices and their metadata (android + ios).

Quick healthcheck of local mobile debugging environment (adb, devices, logs, env, iOS).

Capture a screenshot from an Android device or iOS simulator. Returns device metadata and the screenshot image.

Capture a complete debug snapshot (raw observation layer plus optional derived semantic layer). Returns structured JSON with snapshot_revision, captured_at_ms, snapshot_delta, and loading_state when detectable.

Start streaming logs for a target application on Android or iOS. For Android this uses adb logcat --pid=; for iOS it streams xcrun simctl spawn <device> log stream with a predicate.

Read accumulated log stream entries for the active session.

Stop an active log stream for the session.

Get the current UI hierarchy from an Android device or iOS simulator. Returns a structured JSON representation of the screen content with snapshot metadata and incremental delta signals when available.

Get the currently visible activity on an Android device. Returns package and activity name.

Generate a stable fingerprint representing the current visible screen (activity + visible UI elements).

Purpose: Detect that a screen transition has occurred by waiting for the current fingerprint to differ from a previous fingerprint.

Capabilities:

• Synchronization for uncertain navigation timing

• Detection that something changed on screen

Constraints:

• Does not verify correctness of the resulting state

• Must not be used alone to confirm action success when an applicable expect_* tool exists

• For backend/API activity without a visible UI change, pass the runtime action_type into classify_action_outcome and collect network evidence only if the result remains ambiguous

Recommended Usage:

1. Capture or define the expected outcome

2. Call an action tool

3. Use wait_for_screen_change when transition timing is uncertain

4. Follow with expect_screen when the expected destination is known

Purpose: Wait for a non-navigation UI mutation or in-place update to become stable.

Inputs:

• expected_change (optional): hierarchy_diff, text_change, or state_change

• timeout_ms (optional)

• stability_window_ms (optional)

• scope (optional): screen or subtree

• target (optional): element_id when scope=subtree

Guidance:

• Prefer wait_for_screen_change for navigation transitions.

• Prefer wait_for_ui_change for in-place mutations and non-navigation updates.

• Use the returned snapshot_revision as the observed synchronization point when available.

• Scoped waits return scope-aware stability metadata and a lightweight change summary.

Failure Handling:

• TIMEOUT means the UI did not change in a stable way within the allotted time.

Purpose: Deterministically verify that the intended navigation outcome of an action has occurred.

Inputs:

• fingerprint: preferred exact-match screen fingerprint

• screen: exact semantic screen identifier when a fingerprint is not available

Output Structure:

• success: true when the expected screen matches the observed screen

• observed_screen: current fingerprint and screen identifier

• expected_screen: the expected fingerprint and/or screen identifier

• confidence: 1 for an exact match, otherwise 0

Recommended Usage:

1. Define the expected screen before executing the action

2. Resolve the target element or screen state

3. Call an action tool such as tap_element

4. If needed, wait for transition using wait_for_*

5. Call expect_screen as the final verification step

6. If success=false, treat the outcome as unverified and follow the action tool retry guidance

Verification Guidance:

• Primary and authoritative verification tool for navigation outcomes

• Prefer fingerprint whenever you have one

• Use screen only as a fallback exact match against known identifiers

• Works best when the expected screen identifier is known ahead of time

• If the expected screen is not already known, capture or define it before executing the action

Constraints:

• Returns structured binary success/failure only

• Must not rely on natural-language interpretation or reasoning

Failure Handling:

• success=false means the expected screen was not reached; retry or recover using the action tool's failure strategy

Purpose: Deterministically verify that the intended UI outcome of an action has occurred by confirming a target element is visible.

Inputs:

• selector: required selector used to resolve the target element

• element_id: optional previously resolved element identifier used only as context

Output Structure:

• success: true when the element is visible

• selector: selector used for verification

• element_id: resolved element identifier when available

• element: minimal resolved element info when visible

• failure_code: TIMEOUT or UNKNOWN when verification fails

• retryable: true when failure_code=TIMEOUT

Recommended Usage:

1. Define the expected element state before executing the action

2. Resolve the target element or triggering control

3. Call an action tool such as tap_element

4. If needed, wait for UI availability using wait_for_*

5. Call expect_element_visible as the final verification step

6. If success=false, follow the action tool retry guidance

Verification Guidance:

• Primary and authoritative verification tool for expected element appearance or visibility

• Use this when the screen should stay the same but the UI should reveal or update a specific element

• selector is the primary input; element_id is an optional optimization only

• The tool resolves the selector internally when needed

Constraints:

• Returns structured binary success/failure only

• Must not rely on natural-language interpretation or reasoning

Failure Handling:

• TIMEOUT → retry verification once or retry the action after re-resolving

• UNKNOWN → capture a snapshot and stop

Purpose: Verify a readable UI state property on the currently visible element.

Inputs:

• selector or element_id

• property

• expected

• platform/deviceId (optional)

Supported properties:

• checked, selected, focused, expanded, enabled, text_value, value, raw_value

Verification Guidance:

• Use this when the UI element is visible but its state must also be confirmed

• Prefer the canonical property names above

• The tool compares the normalized readable state and returns the observed value when available

Constraints:

• Returns structured success/failure only

• Does not infer a state when the property is unavailable

Failure Handling:

• ELEMENT_NOT_FOUND → re-resolve the element or wait for UI stabilization

• UNKNOWN → capture a snapshot and stop

Purpose: Adjust a numeric control value with verification.

This is the initial adjustable-control surface for slider-like controls and other controls that expose a numeric value or value_range.

Inputs:

• exactly one of selector or element_id

• property (defaults to "value")

• targetValue

• tolerance (optional; discrete default is 0, continuous default is max(step / 2, 1% of range) when omitted)

• maxAttempts (optional; defaults to 5)

• platform/deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target_state / actual_state / within_tolerance / converged / attempts / adjustment_mode

• target.selector = original selector or element handle

• success = true when the control converges within tolerance

Verification Guidance:

• Prefer direct platform adjustment when the runtime exposes it; then verified increment/decrement actions; use coordinate fallback only as degraded mode

• Use expect_state for the control value readback

• Do not use derived semantic metadata as proof of success

Failure Handling:

• ELEMENT_NOT_FOUND → re-resolve the control

• ELEMENT_NOT_INTERACTABLE → the control cannot be adjusted through the current runtime

• CONTROL_CONVERGENCE_FAILED → the control did not converge within bounded retries or the requested value is out of range

• UNKNOWN → capture a snapshot and stop

Purpose: Resolve elements and/or detect that a UI transition or availability condition has occurred.

Capabilities:

• Deterministic element resolution

• Synchronization when element timing or availability is uncertain

Constraints:

• Does not verify correctness of the resulting state

• Must not be used alone to confirm action success when an applicable expect_* tool exists

Recommended Usage:

1. Use wait_for_ui to resolve an element before acting or to wait for UI readiness

2. Call the action tool

3. If the expected outcome is known, follow with expect_* as final verification

Find a UI element by semantic query (text, content-desc, resource-id, class).

Returns the best match plus resolution metadata when available, including confidence, selection reason, and fallback alternates.

Purpose: Dispatch a tap at specific screen coordinates.

Inputs:

• x, y coordinates

• platform (optional)

• deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { x, y }

• success = true when the tap was dispatched

• failure_code/retryable when dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Resolve coordinates deterministically

2. Call tap

3. If needed, wait for transition using wait_for_*

4. Verify with expect_screen or expect_element_visible depending on the intended outcome

Verification Guidance:

• Prefer tap_element over tap when an element can be resolved

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Use expect_screen for navigation and expect_element_visible for local UI changes

• Do not use wait_for_* alone as final verification when an applicable expect_* tool exists

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → capture a snapshot and stop

Purpose: Tap a previously resolved UI element using its elementId.

Inputs:

• elementId: a resolved UI element identifier returned by wait_for_ui

Output Structure:

• action_id: unique timestamp-based action identifier

• timestamp: ISO 8601 timestamp for the action attempt

• action_type: "tap_element"

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector: original target handle ({ elementId })

• target.resolved: minimal resolved element info used for the tap

• success: true when the tap was dispatched

• failure_code: present when success=false

• retryable: present when failure_code exists

• ui_fingerprint_before/ui_fingerprint_after: optional fingerprints captured around the action

Recommended Usage:

1. Resolve the target with wait_for_ui or another deterministic resolver

2. Call tap_element

3. If needed, wait for transition using wait_for_*

4. Verify outcome using expect_*

   • use expect_screen when navigation is expected

   • use expect_element_visible when the UI change is local

5. If verification fails, inspect failure_code and follow the retry strategy below

Verification Guidance:

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Prefer expect_screen for navigation or modal transitions

• Prefer expect_element_visible when the tap should reveal or update a specific element

• wait_for_* may be used for resolution and synchronization, but not as the final verification step when an applicable expect_* tool exists

• Do not treat tap_element.success as outcome success; it only means the tap was executed

Failure Handling:

• STALE_REFERENCE → re-resolve the element, then retry

• ELEMENT_NOT_INTERACTABLE → wait or refine the target, then retry

• UNKNOWN → capture a snapshot and stop

This tool reports execution success only. Verification must be done with a separate expect_* tool.

Purpose: Dispatch a swipe gesture on Android or iOS.

Inputs:

• start and end coordinates

• duration

• platform/deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { x1, y1, x2, y2, duration }

• success = true when the swipe was dispatched

• failure_code/retryable when dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Determine swipe coordinates

2. Call swipe

3. If needed, wait for transition using wait_for_*

4. Verify with expect_screen or expect_element_visible when a deterministic outcome is expected

Verification Guidance:

• Swipe outcomes are less predictable; choose the most specific verifier available for the intended effect

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Do not use wait_for_* alone as final verification when an applicable expect_* tool exists

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → capture a snapshot and stop

Purpose: Scroll until a target element becomes visible.

Inputs:

• platform

• selector

• direction, maxScrolls, scrollAmount, deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = original selector

• target.resolved = minimal resolved element info when found

• success = true when scrolling produced a visible target element

• failure_code/retryable when the target was not reached

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Resolve the target selector

2. Call scroll_to_element

3. If needed, wait for UI stabilization using wait_for_*

4. Verify with expect_element_visible when the expected element visibility is known

5. If success=false, follow failure handling before retrying

Verification Guidance:

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Use expect_element_visible when you need an explicit post-scroll confirmation

• Do not use wait_for_* alone as final verification when an applicable expect_* tool exists

Failure Handling:

• NAVIGATION_NO_CHANGE → adjust scroll direction or stop

• TIMEOUT → retry with refined selector or larger scroll budget

• UNKNOWN → capture a snapshot and stop

Purpose: Type text into the currently focused Android input field.

Inputs:

• text

• platform/deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { text }

• success = true when text input was dispatched

• failure_code/retryable when dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Resolve or focus the target input first

2. Call type_text

3. If needed, wait for UI stabilization using wait_for_*

4. Verify with expect_element_visible or expect_screen, depending on the intended outcome

Verification Guidance:

• Prefer verifying the next expected element or screen state instead of inferring success from the text action alone

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Do not use wait_for_* alone as final verification when an applicable expect_* tool exists

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → re-focus the input or capture a snapshot

Purpose: Dispatch the Android Back action.

Inputs:

• platform/deviceId (optional)

Output Structure:

• action_id, timestamp (ISO 8601), action_type

• lifecycle_state: post-dispatch lifecycle state (pending_verification or failed)

• source_module: runtime source of the action envelope

• target.selector = { key: "back" }

• success = true when the back action was dispatched

• failure_code/retryable when dispatch fails

• ui_fingerprint_before/ui_fingerprint_after when available

Recommended Usage:

1. Call press_back

2. If needed, wait for transition using wait_for_*

3. Verify with expect_screen when a known destination is expected

4. If verification fails, retry once or recover explicitly

Verification Guidance:

• Back outcomes can vary by screen, so verify against the intended destination when possible

• Follow RESOLVE → ACT → WAIT (if needed) → EXPECT

• Do not use wait_for_* alone as final verification when an applicable expect_* tool exists

Failure Handling:

• TIMEOUT → retry once

• UNKNOWN → capture a snapshot and stop

Classify the outcome of the most recent action into exactly one of: success, no_op, backend_failure, ui_failure, unknown.

Use the runtime action result's action_type as actionType so the classifier can distinguish local-state actions from side-effect actions. Use this when the intended outcome is not already fully verified by the UI signal alone. For backend/API activity, compare get_screen_fingerprint before and after the action and call get_network_activity immediately after the action if the outcome is still ambiguous.

HOW TO GATHER INPUTS before calling:

1. Call wait_for_screen_change or compare get_screen_fingerprint before/after — set uiChanged accordingly.

2. If you checked for a specific element with wait_for_ui, set expectedElementVisible.

3. Pass actionType from the action response when available.

4. Only provide networkRequests if you already collected them or want to classify a side-effect action with backend evidence.

RULES (applied in order — stop at first match):

1. If uiChanged=true OR expectedElementVisible=true → outcome=success

2. If actionType is missing → outcome=unknown

3. If any request has status=failure or retryable → outcome=backend_failure

4. If actionType maps to a local-state action → outcome=no_op; prefer state-based verification and avoid default network fallback

5. If actionType maps to a side-effect action and no networkRequests were supplied → outcome=unknown

6. If no requests returned → outcome=no_op

7. If all requests succeeded → outcome=ui_failure

8. Otherwise → outcome=unknown

BEHAVIOUR after outcome:

• success → continue

• no_op → retry with richer state verification or re-resolve the element

• backend_failure → stop and report the failing endpoint

• ui_failure → stop and report failure

• unknown → take one recovery step (e.g. capture_debug_snapshot), then stop

Returns structured network events captured from platform logs since the last action.

Call this immediately after an action when you want backend evidence for a side-effect flow, only if the result is still ambiguous. Do not call more than once per action.

Events are filtered to significant (non-background) requests only. Each event includes endpoint, method, statusCode, networkError, status, and durationMs.

status values:

• success: HTTP 2xx or request detected with no error signal

• failure: HTTP 4xx

• retryable: HTTP 5xx, network error (timeout, dns_error, tls_error, etc.)

Returns { requests: [], count: 0 } when no credible network signals are found.