aua_locate
Locate UI elements missing from Android accessibility trees by visual description; returns a masked screenshot-based bounding box and tap point for canvas, games, WebView images, or unlabeled icons.
Instructions
Read-only Jev Box location for elements the accessibility tree lacks (canvas, games, WebView images, icons without labels); small text such as prices or tags is unreliable, so select text with selectors. Sends one masked screenshot of region (default full display) to TypeSafe, so it needs Jev plus the separate images opt-in (允许发送截图(视觉定位) in screen settings, or ANDROID_USE_JEV_IMAGES=1); otherwise jev_images_disabled. Refuses password/OTP screens and paused devices. One request asks presence, match count and a box; status is located, not_found, ambiguous, low_confidence or masked; a box spanning several tree labels is ambiguous with reason and tree_candidates. box=[x,y,width,height] and tap=[x,y] are display pixels; tap is set only when located. masks blank sensitive areas before upload. Prefer trees/selectors when they work; a located box is a model judgment, not verification.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| masks | No | ||
| region | No | ||
| package | Yes | ||
| description | Yes | ||
| min_presence | No | ||
| min_confidence | No | ||
| timeout_seconds | No |