Skip to main content
Glama
expoli

android-phone-mcp-server

by expoli

locate

Find an on-screen element from its description when the accessibility tree and OCR fail. Uses a vision-language model to return its bounding box.

Instructions

Locate a described element via the VLM grounding layer (read-only).

The THIRD perception layer (tree -> OCR -> VLM): when the accessibility tree and OCR cannot resolve a target on a tree-failed screen (Flutter/ Unity/games), this asks a vision-language model for the element's bounding box on the current screenshot.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
deviceNoADB serial; omit with a single connected device.
targetYes{"by": "text"|"desc", "value": "<description>"} — describe the element you are looking for (e.g. "the blue download button").

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv0.4.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full transparency burden. It discloses the read-only nature and the underlying behavior ('asks a vision-language model for the element's bounding box on the current screenshot'). This is solid, though it stops short of describing potential failure or nondeterminism.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the main action and safety hint ('read-only'), immediately followed by the key contextual detail. No fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a reasonably simple read-only locate tool with an output schema, the description provides enough context: when to use it, what it does, and on what input it operates. It leaves out error/not-found behavior, but that is likely covered by the output schema; still, a mention of fallback behavior would make it more complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers 100% of parameters with detailed descriptions, including a concrete example for target. The description adds contextual framing but no additional parameter-level meaning, so the schema-driven baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the action ('Locate a described element via the VLM grounding layer') and frames it as the 'THIRD perception layer (tree -> OCR -> VLM)', clearly distinguishing it from sibling tools like get_screen and verify_element. It names the exact resource (element) and mechanism (VLM grounding).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance: 'when the accessibility tree and OCR cannot resolve a target on a tree-failed screen (Flutter/Unity/games)'. The layer ordering tree -> OCR -> VLM also clarifies when not to use it, making the usage context unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.