describe_ui
Read the live Android UI hierarchy to get exact element coordinates for tapping, and filter by text, ID, or clickable state to verify what's on screen.
Instructions
Read the on-screen UI hierarchy as a list of elements, each with its text, content_desc, resource_id, class, clickable flag, pixel bounds, and a precomputed center in TRUE DEVICE PIXELS. This is your source of truth for AIMING: pass an element's center straight to tap. Never guess coordinates from the screenshot (it is downscaled and you will miss). The response header states the FOCUSED WINDOW (if it's a system overlay — biometric prompt, permission dialog — the elements belong to that overlay, not your app) and how many nodes the filter hid. Default filter keeps labelled/clickable/id-carrying elements minus redundant wrappers; filter="clickable" returns only tap targets (much smaller); filter="all" returns every bounded node — the only mode where absence proves an element isn't in the CURRENT VIEWPORT's hierarchy. Every mode is viewport-scoped: Android omits a ScrollView's off-screen children from the accessibility tree entirely, so scroll before concluding an element does not exist, and never use this to COUNT a long list (it will undercount) — render_stats reports the whole view tree. Canvas-drawn (RN/Flutter/Skia) content appears in NO mode.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | Case-insensitive substring to match against text, content_desc, and resource_id — return only matching elements. The cheap way to ask 'is X on this screen?'. Combine with filter='all' to prove absence definitively. | |
| filter | No | What to include: 'auto' (default — elements with text, content_desc, resource_id, or clickable; identical-bounds label-less wrappers dropped), 'clickable' (tap targets only, the smallest view), or 'all' (every bounded node, unfiltered — use to PROVE an element is absent from the CURRENT VIEWPORT). 'all' is still viewport-scoped: Android omits off-screen ScrollView children from the accessibility tree, so scroll before concluding absence, and use render_stats (not this) to count a long list. | |
| serial | No | Target device serial (adb -s). Optional when exactly one device is attached. | |
| compact | No | Return one line per element (center, bounds, flags, labels) instead of JSON — ~10x fewer tokens, same aiming information. Use for repeated look-drive loops and geometry work. | |
| package | No | Optional package expected to own the focused window. If another app or SystemUI owns focus, the response calls that out explicitly. |