Android Use

Overview Schema Source Related Servers Discussions Admin

aua_locate

Read-onlyIdempotent

Finds on-screen elements the accessibility tree can't describe, like canvas buttons, game targets, or unlabeled WebView icons, by locating them in a masked screenshot and returning a tap-ready box.

Instructions

Read-only Jev Box location for elements the accessibility tree lacks (canvas, games, WebView images, icons without labels). Sends one masked screenshot of region (default full display) to TypeSafe, so it needs Jev plus the separate images opt-in (允许发送截图（视觉定位） in screen settings, or ANDROID_USE_JEV_IMAGES=1); otherwise jev_images_disabled. Refuses password/OTP screens and paused devices. One request asks presence, match count and a box; status is located, not_found, ambiguous, low_confidence or masked. box=[x,y,width,height] and tap=[x,y] are display pixels; tap is set only when located. masks blank sensitive areas before upload. Prefer trees/selectors when they work; a located box is a model judgment, not verification.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "package": "<package>",
      "description": "<description>"
    },
    "name": "aua_locate"
  }
}

Input Schema

Name	Required	Description	Default
`masks`	No
`region`	No
`package`	Yes
`description`	Yes
`min_presence`	No
`min_confidence`	No
`timeout_seconds`	No

Schema Changelog

Changes observed during successful MCP inspections.

Addedv0.6.0Nov 5, 2026

TDQS

A4.3/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the read-only/idempotent/open-world profile, so this disclosure is substantial: it reveals that a masked screenshot is uploaded to a third-party service (TypeSafe), the exact opt-in gate and its error code jev_images_disabled, refusal conditions (password/OTP screens, paused devices), and the full status vocabulary (located, not_found, ambiguous, low_confidence, masked). It even warns that a located box is a model judgment rather than verification, which is exactly the kind of caveat annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads purpose and scope, then preconditions, then return semantics — a sound ordering for a single dense paragraph. It is long and occasionally run-on (the parenthetical opt-in string with Chinese UI label plus env var), but nearly every clause carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, no-output-schema vision tool, the description covers the privacy/upload model, gating, failure modes, and the return fields (status, box, tap). What is missing is any guidance on the numeric threshold parameters and the package/description inputs, which an agent must guess when tuning a request.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden. It explains region (defaults to the full display), masks (blanked before upload), and the shape of the returned box/tap rather than the input. The four remaining inputs — package, description, min_presence, min_confidence, timeout_seconds — get no explanation of units, accepted values, or how they interact with the returned status, leaving a real gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Read-only Jev Box location') and immediately scopes it to elements the accessibility tree cannot express (canvas, games, WebView images, unlabeled icons). This cleanly separates it from the tree/selector siblings such as aua_find without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear when-to-use rule (elements missing from the accessibility tree) and an explicit when-not rule ('Prefer trees/selectors when they work'). Preconditions are spelled out too (images opt-in flag or env var, no password/OTP screens, no paused devices). It stops just short of naming the concrete alternative tool (e.g. aua_find), so routing is inferred rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.