describe_ui
Read the current UI hierarchy as a list of tappable elements with exact pixel centers, so you can aim taps accurately instead of guessing from a downscaled screenshot.
Instructions
Read the on-screen UI hierarchy as a list of elements, each with its text, content_desc, resource_id, class, clickable flag, pixel bounds, and a precomputed center in TRUE DEVICE PIXELS. This is your source of truth for AIMING: pass an element's center straight to tap. Never guess coordinates from the screenshot (it is downscaled and you will miss). The response header states the FOCUSED WINDOW (if it's a system overlay — biometric prompt, permission dialog — the elements belong to that overlay, not your app) and how many nodes the filter hid. Default filter keeps labelled/clickable/id-carrying elements minus redundant wrappers; filter="clickable" returns only tap targets (much smaller); filter="all" returns every bounded node — the only mode where absence proves an element isn't in the hierarchy. Canvas-drawn (RN/Flutter/Skia) content appears in NO mode.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | Case-insensitive substring to match against text, content_desc, and resource_id — return only matching elements. The cheap way to ask 'is X on this screen?'. Combine with filter='all' to prove absence definitively. | |
| filter | No | What to include: 'auto' (default — elements with text, content_desc, resource_id, or clickable; identical-bounds label-less wrappers dropped), 'clickable' (tap targets only, the smallest view), or 'all' (every bounded node, unfiltered — use to PROVE an element is absent from the hierarchy). | |
| serial | No | Target device serial (adb -s). Optional when exactly one device is attached. | |
| compact | No | Return one line per element (center, bounds, flags, labels) instead of JSON — ~10x fewer tokens, same aiming information. Use for repeated look-drive loops and geometry work. | |
| package | No | Optional package expected to own the focused window. If another app or SystemUI owns focus, the response calls that out explicitly. |