Skip to main content
Glama

aua_locate

Read-onlyIdempotent

Locate UI elements missing from Android accessibility trees by visual description; returns a masked screenshot-based bounding box and tap point for canvas, games, WebView images, or unlabeled icons.

Instructions

Read-only Jev Box location for elements the accessibility tree lacks (canvas, games, WebView images, icons without labels); small text such as prices or tags is unreliable, so select text with selectors. Sends one masked screenshot of region (default full display) to TypeSafe, so it needs Jev plus the separate images opt-in (允许发送截图(视觉定位) in screen settings, or ANDROID_USE_JEV_IMAGES=1); otherwise jev_images_disabled. Refuses password/OTP screens and paused devices. One request asks presence, match count and a box; status is located, not_found, ambiguous, low_confidence or masked; a box spanning several tree labels is ambiguous with reason and tree_candidates. box=[x,y,width,height] and tap=[x,y] are display pixels; tap is set only when located. masks blank sensitive areas before upload. Prefer trees/selectors when they work; a located box is a model judgment, not verification.

Input Schema

NameRequiredDescriptionDefault
masksNo
regionNo
packageYes
descriptionYes
min_presenceNo
min_confidenceNo
timeout_secondsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.7.0

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the readOnly/openWorld/idempotent annotations: it discloses that a masked screenshot leaves the device to a third party (TypeSafe), the exact opt-in toggle and env var, the jev_images_disabled failure, refusal of password/OTP screens and paused devices, and the five possible statuses. The remaining gap is that it does not say how long a request may take or what a masked upload costs in latency, though timeout_seconds hints at it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose and the strongest routing rule in the first clause, then prerequisite, then refusals, then output shape. The sentences are dense to the point of being comma-chained and some parenthetical detail (the Chinese toggle label) could be trimmed, but almost every clause carries usable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an open-world vision tool with no output schema, the description covers return shape (box, tap, status enum, ambiguous reason with tree_candidates), pixels vs. logical coordinates, and the honesty caveat that a located box is a model judgment rather than verification. Only the unlabeled numeric thresholds keep it short of complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% across 7 parameters, so the description has to carry semantics and only partly does: it explains region's default ('default full display'), masks ('masks blank sensitive areas before upload'), and the output box/tap pixel convention. It says nothing about min_presence, min_confidence, timeout_seconds, or the package pattern, which are the tuning knobs an agent would most likely set wrong.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — read-only visual location of elements — and scopes it precisely to what the accessibility tree lacks (canvas, games, WebView images, unlabeled icons). That scope statement alone separates it from tree-based siblings like aua_find and from aua_screen_action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when not to use it ('small text such as prices or tags is unreliable, so select text with selectors') and closes with 'Prefer trees/selectors when they work'. It also names the prerequisite condition for use (Jev plus the images opt-in) and the fallback failure code when unmet.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.