tap_with_fallback
Sends a raw coordinate tap using native backend with Maestro fallback for WKWebView overlays where visual position differs from DOM hit-test. Verifies tap success via accessibility tree changes or screenshot delta.
Instructions
Sends a raw coordinate tap via the native backend (idb if installed, else a Maestro tapOn-point fallback). Useful for WKWebView game overlays where visual position differs from the DOM hit-test position. The Maestro fallback needs an app context: pass bundleId, or the foreground app is auto-detected. VERIFICATION: 'ok' is decided primarily by a change in the native accessibility element set before/after the tap (stable under animation/video). When no native backend is present it falls back to a screenshot byte-size delta (weak — animation can flip it). The result's oracle field reports which was used ('a11y-change' | 'screenshot-bytesize' | 'unverified'). For WebView-rendered targets the a11y tree won't change → oracle:'unverified'; confirm via webview_inspect. offsetStep defaults to 0 (tap the exact point); set it >0 only to deliberately probe nearby y-offsets on retry.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"udid": "<udid>",
"x": "<x>",
"y": "<y>"
},
"name": "tap_with_fallback"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| udid | Yes | Simulator / device UDID | |
| x | Yes | X coordinate in logical points | |
| y | Yes | Y coordinate in logical points | |
| bundleId | No | App bundle id for the Maestro fallback (ignored when idb is present). Auto-detected if omitted. | |
| maxRetries | No | Maximum tap attempts (default 3) | |
| offsetStep | No | Opt-in Y offset step in px applied per retry (default 0 = always tap the exact point; no blind walk). |