android-phone-mcp-server
This server provides semantic control and observation of Android devices via ADB, using three perception layers (accessibility tree, OCR, and VLM) for reliable UI interaction. All actions return structured results with verification evidence and machine-readable errors. Write operations are disabled by default (enable via environment variables); execution budgets prevent runaway agents. Multi-device support is built in.
Device Management: List connected devices and get device info (model, manufacturer, Android version, resolution, density).
Screen Observation: Capture compact or full text-mode snapshots with stable element IDs; get screen hashes for change detection; diff current vs cached screen state; locate elements via VLM when accessibility tree and OCR fail.
Semantic Interactions: Tap by text, content description, element ID, or coordinates; swipe in any direction with short/long distance; scroll exactly one viewport page; type text into focused or targeted fields; scroll until a target appears; aggregate matching elements across multiple screens (including OCR) with
smart_scroll; open apps by name or package; deep-link into system settings panels; simulate system key presses (back, home, volume, etc.).Verification & Waiting:
wait_forpolls until an element appears or disappears;verify_elementchecks existence and optionally a text predicate; all actions return verification evidence (changed elements, screen hash).Session Management: Reset session caches and execution budget counters.
Key Features: Read-only by default; execution budgets on action count/time; structured errors with hints; three-layer perception; full verification feedback for agent reliability.
Provides semantic control of Android devices via ADB, including tap, swipe, scroll, text input, opening apps and settings, and screen observation with verification.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@android-phone-mcp-serverOpen the Settings app and go to the About phone page"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
android-phone-mcp
代码 Agent 无关的 Android 控制 MCP Server:语义动作 + 验证闭环 + 融合感知。 Any MCP client (Claude Code / Cursor / Cline / 自研 Agent) can control an Android device through semantic tools — no coordinate guessing, every action returns verification evidence.
✅ Phase 0(语义动作 + 验证闭环)/ Phase 1(OCR 融合感知 + 校验工具集 + 多设备并行)/ Phase 2(VLM 视觉融合 + 执行预算 + 增量 diff + 评测)均已完成真机验收。 完整设计见
android-phone-mcp-server-设计文档.md与openspec/。
快速开始(开发环境)
# 1. 创建虚拟环境并安装(uv;国内网络请配置镜像)
uv venv .venv
export UV_DEFAULT_INDEX=https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple # 可选
uv pip install -e ".[dev]"
# 2. 连接 WiFi adb 设备
adb connect <phone-ip>:<port> # 例如 192.168.1.15:39455
adb devices # 确认 device 状态
# 3. 启用开发写权限(默认只读!)
cp .env.example .env
# 4. 启动 MCP server(stdio)
.venv/bin/android-phone-mcp --stdio
# 5. 常用检查
.venv/bin/android-phone-mcp --show-config # 查看生效配置
.venv/bin/android-phone-mcp --list-tools # 列出全部 18 个工具作为 MCP 客户端接入(Python 示例)
import asyncio
from fastmcp import Client
from android_phone_mcp.config import Config
from android_phone_mcp.server import create_server
async def main():
server = create_server(config=Config(allow_write=True)) # 开发期开写
async with Client(server) as client:
r = await client.call_tool("open_settings", {"panel": "about_phone"})
print(r.data) # {executed, screen_changed, changed_elements[], screen_hash}
asyncio.run(main())任意 MCP 客户端(Claude Code / Cursor / Cline / 自研 Agent)均可通过 MCP 协议接入;错误以结构化 JSON 返回(如 WRITE_DISABLED / SELECTOR_AMBIGUOUS + candidates[]),模型可直接读取提示继续操作。
Related MCP server: airi-android
安全模型
开关 | 环境变量 | 默认 |
写操作(tap/输入/安装等) |
| off(只读) |
任意 |
| off |
每动作超时 |
| 30s |
配置优先级:环境变量 > config.yaml > 内置默认。.env 仅用于本地开发一键开写。
工具清单(18 个)
类别 | 工具 |
设备 |
|
观察 |
|
动作 |
|
等待 |
|
会话/护栏 |
|
校验工具(阶段 1 新增):
wait_for(target, state=present|absent, timeout=10):阻塞等待元素出现/消失(加载动画)verify_element(target, text_predicate?):断言元素存在/缺失/文本匹配,结构化布尔diff_state():当前 vs 上次快照的增量 diff(added/removed/changed + 双 hash)smart_scroll(target):滚动聚合多屏全部命中(含 OCR 感知),一次调用返回
OCR 融合感知(阶段 1)
无障碍树对 Flutter/Unity/游戏等自渲染引擎失效时,get_screen 自动回退 OCR 文本层:识别文本以 ocr- 前缀伪元素合并进紧凑快照(共享 id 空间),tap/type_text 可直接定位;tap 前会重新 OCR 确认位置(OCR_STALE 兜底)。
# 安装 OCR 引擎(可选 extra,默认不装)
uv pip install -e ".[ocr]" # 或 pip install android-phone-mcp[ocr]
# 引擎:rapidocr_onnxruntime(py3.14 无 paddlepaddle wheel 时的替代,中文识别佳)配置 | 默认 | 说明 |
| true | OCR 兜底开关 |
| 3 | 树可交互元素低于该值触发 OCR |
VLM 视觉融合(阶段 2 · 第三感知层)
当无障碍树与 OCR 都无法定位目标时(抽象图形界面/游戏 UI),可选 VLM grounding(OpenAI 兼容视觉端点,如 Qwen2.5-VL via vLLM/Ollama)作为最终兜底:
locate(target):按文本/描述返回目标位置{found, position, box}自动兜底:语义工具(tap/scroll_until 等)目标未命中且 VLM 可用时,自动 VLM 定位,
vlm-伪元素进快照(共享 id 空间);tap 前重确认(VLM_STALE兜底)SoM 编号截图通道:vision 模型可直接看编号截图操作
# 安装 VLM 引擎(可选 extra,默认不装)
uv pip install -e ".[vlm]" # 或 pip install android-phone-mcp[vlm]配置 | 默认 | 说明 |
| true | VLM 兜底开关 |
| 空 | OpenAI 兼容端点(如 |
| 空 | 模型名(如 |
| 空 | 本地端点可留空 |
VLM 为服务端能力——调用模型不需要视觉;未配置端点时结构化降级(
VLM_UNAVAILABLE),不影响树/OCR 路径。
执行预算(阶段 2 · 防失控护栏)
会话级动作数/时长上限,防止 Agent 死循环/狂点:
配置 | 默认 | 说明 |
| 0(关) | 最大写动作数,超出拒绝 |
| 0(关) | 自首个动作起最大时长 |
超限后写工具返回 EXECUTION_BUDGET_EXCEEDED(含 limit 与恢复提示);调用 reset_session 清除快照缓存并重置预算:
tap(...) # ok
tap(...) # 第 N+1 次 -> {error: "EXECUTION_BUDGET_EXCEEDED", limit: "actions", hint: "..."}
reset_session() # {ok: true, budget: {action_count: 0, ...}}
tap(...) # ok(预算已重置)AndroidWorld 子集评测(阶段 2)
# 运行评测(需要 WiFi adb 设备)
ANDROID_TEST_DEVICE=<serial> .venv/bin/python -m eval.runner
# 输出 eval/report.json:每任务 {task, success, steps, error/reason}WiFi adb 注意事项与重连流程
WiFi adb 注意事项与重连流程
调试设备通过 WiFi adb 连接时,请勿在测试中切换 Wi-Fi 开关——关闭瞬间手机会断网,adb 连接随之断开
IP:端口会随重连变化;灭屏/省电可能导致连接离线
重连流程:
# 1. 确认离线
adb devices # 设备消失或显示 offline
# 2. 断开旧连接并重新 connect(手机端需保持"无线调试"开启)
adb disconnect
adb connect <phone-ip>:<port>
# 3. 验证
adb devices # 应显示 device(非 offline)
adb -s <serial> shell getprop ro.product.model # 输出模型号即正常若
connect后反复 offline:检查手机无线调试端口是否变化、与电脑是否同网段、防火墙是否放行 5555 段端口设备池在每次工具调用前做 health check,连接失效时返回结构化错误,不会卡死调用
验收用例(Phase 0 三用例,均为网络无关操作)
用例 | 操作链 | 断言 |
① 关于手机 |
| 读到设备信息元素(手机名称/存储空间/运行内存/电池) |
② 打开 App |
|
|
③ 表单填写 | 定位输入框 → | 搜索结果出现(如输入 wifi 出现 WLAN 相关项) |
验收用例固化为 |
开发与测试
.venv/bin/pytest # 单测(无设备,集成用例自动跳过)
.venv/bin/pytest tests/test_integration.py -v # 验收用例(需设备)
ANDROID_TEST_DEVICE=192.168.1.15:39455 .venv/bin/pytest # 全量含真机探针路线图
阶段 0 MVP:FastMCP + uiautomator2 + 语义动作 + 屏幕哈希 + 验证闭环 ✅
阶段 1 通用性:OCR 融合感知 + 校验工具集(wait_for/verify_element/diff_state/smart_scroll)+ 多设备并行 ✅
阶段 2 完整版:VLM 视觉融合(SoM) + 执行预算 + 增量 diff + AndroidWorld 子集评测 ✅ ← 当前
后续:PyPI 发布(uv publish,token 就绪后)、端侧 companion App(独立 Android 工程)
Available Tools
18 toolsdiff_stateA
Diff the current screen against the last cached snapshot.
Returns added/removed/text-changed element lists (stable-id diff) plus both screen hashes — the explicit counterpart of the automatic evidence returned by every action tool.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that it compares against the last cached snapshot, returns added/removed/text-changed element lists (stable-id diff), and both screen hashes. It also clarifies its relationship to automatic evidence. It does not explicitly state it is read-only or what happens if no snapshot exists, but 'diff' strongly implies non-mutating behavior, and the output description adds context beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, tightly packed with meaningful information: what it does, what it returns, and its relationship to other tools. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core behavior and return values, and an output schema exists to detail the exact structure. It doesn't mention edge cases (e.g., no snapshot), but given the tool's simplicity and available schema, this is sufficient. Could be a 5, but a note about snapshot prerequisites would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter (device) is fully documented in the schema. The description does not mention parameters, which is fine because the schema already handles it. Baseline 3 applies; no extra value needed from description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Diff the current screen against the last cached snapshot.' It uses a specific verb ('diff') and resource ('screen' vs 'cached snapshot'), and distinguishes itself by mentioning it is the 'explicit counterpart of the automatic evidence returned by every action tool.' This separates it from sibling tools like get_screen or get_screen_hash.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context by calling itself the 'explicit counterpart of the automatic evidence returned by every action tool,' implying use when explicit diff evidence is needed. However, it does not explicitly state when not to use it or name alternative tools. This is clear context without exclusions, meriting a 4.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_device_infoA
Read device facts: model, manufacturer, Android version, resolution, density.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial (WiFi form ip:port or USB serial). Omit when a single device is connected; with several devices it must be set. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. 'Read' indicates a non-destructive operation, but the description does not disclose potential behaviors like how it handles multiple connected devices or what the output structure is. While the schema mentions the device parameter, the description alone offers limited behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, highly readable sentence that leads with the action and lists the key facts. It is optimally concise with no filler or redundant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple and an output schema exists, so the description does not need to explain return values. It covers the core purpose adequately. A minor gap is the lack of explicit handling for the optional device parameter, but the schema provides that context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description fully covers the single parameter (device), so the baseline is 3. The description does not add any additional parameter semantics beyond what the schema already provides, but no extra information is necessary given the high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the exact action ('Read') and the resource ('device facts') along with a concrete list of attributes (model, manufacturer, Android version, resolution, density). This clearly distinguishes the tool from sibling action tools like tap or swipe.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when device properties are needed, but it does not explicitly state when to use this tool over alternatives or mention any exclusions. The simple read-only nature makes the context clear, yet no direct guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_screenA
Capture a compact text-mode snapshot of the current screen.
Returns elements as [id] class "label" kind@(x,y) lines plus the
screen hash. Elements carry stable ids you can reference in tap().
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | "summary" (interactive elements only, ~14x smaller) or "full" (every labeled/interactive element). | summary |
| device | No | ADB serial; omit with a single connected device. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool returns a text-mode snapshot with elements formatted as `[id] class "label" kind@(x,y)` plus the screen hash, and that ids are stable for later use. This gives meaningful insight into the tool's output behavior, though it stops short of detailing prerequisites or edge cases like device connection errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, about 40 words. It front-loads the core purpose and then details the return format and the relevance of stable ids. Every sentence adds value without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only 2 optional parameters with full schema descriptions and an output schema (per context signals). The description supplements this by explaining the output line format and the screen hash, which is useful beyond the schema. It is complete enough for an agent to correctly invoke the tool, though it could optionally mention default mode behavior or device‑related failures.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for both parameters (mode and device), each with clear descriptions. The tool description adds no additional parameter semantics beyond the schema, so the baseline of 3 applies. The mention of 'compact' in the description aligns with mode defaults but does not extend the schema's meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action: 'Capture a compact text-mode snapshot of the current screen.' This distinguishes it from siblings like get_screen_hash (which only returns a hash) and tap (for interactions). The return format is explicitly described, reinforcing the tool's unique role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it returns stable element ids that can be used with tap(). This implies when to use the tool (when you need to inspect or interact with screen elements). It does not explicitly mention alternatives or exclusions, but the sibling context and the stable-id feature effectively steer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_screen_hashA
Return a cheap fingerprint of the current screen (hash of the snapshot).
Call this before and after an action to detect whether the screen changed, without refetching the full snapshot.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | "summary" (default) or "full". | summary |
| device | No | ADB serial; omit with a single connected device. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of disclosure. It discloses that the operation is cheap and returns a hash-based fingerprint, conveying read-only-like behavior. However, it doesn't detail edge cases or side effects, but for a simple hash tool this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the core purpose front-loaded and the usage case following immediately. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the presence of an output schema, and the clear use-case explanation, the description is complete. It doesn't need to explain return values since the output schema exists, and the behavioral context is adequately covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not add any clarification about the 'mode' parameter (summary vs full) beyond what the schema already states, leaving the agent without additional semantic guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a cheap fingerprint of the current screen (a hash of the snapshot), using a specific verb and resource. It also distinguishes itself from get_screen by explicitly noting it avoids refetching the full snapshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly guides the agent to call this before and after an action to detect screen changes, and points out the alternative of not refetching the full snapshot, which implies get_screen as the alternative for full content. This gives clear context on when to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_devicesA
List all adb-connected Android devices as {serial, connection_type}.
Returns an empty list (never an error) when no device is connected.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses a key behavior: returns an empty list (never an error) when no device is connected. It also specifies the return format, which adds useful context. It doesn't mention prerequisites like ADB being installed, but for a simple tool this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information. The description front-loads the core action and result, and then adds the exception behavior in a single clause. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple zero-parameter list tool, the description is complete. It specifies the output schema, the edge case of no devices, and the error behavior. The presence of an output schema further reduces the need for more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description adds value by explaining the output format, which is more relevant than parameter details. There's no parameter ambiguity to resolve.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('List') and resource ('all adb-connected Android devices'), and specifies the output structure ({serial, connection_type}). This distinguishes it from sibling tools like tap or get_device_info, which operate on individual devices rather than enumerating them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for device discovery/enumeration without explicitly naming alternatives. Since no other sibling tool lists devices, it's clear when to use this. It lacks explicit 'use this when' statements but the context is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
locateA
Locate a described element via the VLM grounding layer (read-only).
The THIRD perception layer (tree -> OCR -> VLM): when the accessibility tree and OCR cannot resolve a target on a tree-failed screen (Flutter/ Unity/games), this asks a vision-language model for the element's bounding box on the current screenshot.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc", "value": "<description>"} — describe the element you are looking for (e.g. "the blue download button"). |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full transparency burden. It discloses the read-only nature and the underlying behavior ('asks a vision-language model for the element's bounding box on the current screenshot'). This is solid, though it stops short of describing potential failure or nondeterminism.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the main action and safety hint ('read-only'), immediately followed by the key contextual detail. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a reasonably simple read-only locate tool with an output schema, the description provides enough context: when to use it, what it does, and on what input it operates. It leaves out error/not-found behavior, but that is likely covered by the output schema; still, a mention of fallback behavior would make it more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers 100% of parameters with detailed descriptions, including a concrete example for target. The description adds contextual framing but no additional parameter-level meaning, so the schema-driven baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Locate a described element via the VLM grounding layer') and frames it as the 'THIRD perception layer (tree -> OCR -> VLM)', clearly distinguishing it from sibling tools like get_screen and verify_element. It names the exact resource (element) and mechanism (VLM grounding).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance: 'when the accessibility tree and OCR cannot resolve a target on a tree-failed screen (Flutter/Unity/games)'. The layer ordering tree -> OCR -> VLM also clarifies when not to use it, making the usage context unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
open_appA
Launch an installed app by friendly name or package name.
Unknown/not-installed apps return a structured APP_NOT_INSTALLED error (with a hint to pass the exact package name).
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| app_name | Yes | e.g. "设置", "settings", "browser", or "com.android.settings". | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that unknown/not-installed apps yield a structured APP_NOT_INSTALLED error with a suggestion to use the exact package name. This adds value beyond the input schema, which only describes parameters. With no annotations, it covers an important failure mode, though it stops short of other behavioral details such as foregrounding or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, followed by a concrete error-handling detail. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 3 parameters and an output schema, the description covers the main action and a key error case. However, it does not mention session_id or differentiate from open_settings, leaving some ambiguity in tool selection. Overall, the description is sufficient but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes app_name and device; the description's 'friendly name or package name' paraphrases app_name's examples. session_id remains undocumented at 67% coverage, and the description does not clarify it. The added semantic value over the schema is minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Launch an installed app by friendly name or package name,' which clearly states the action and resource. It distinguishes from sibling open_settings by implying a general app launcher, though it doesn't explicitly name that sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the usage scenario: to launch an installed app. However, it provides no explicit when-not-to-use guidance and does not mention alternatives like open_settings. The error hint is about failure recovery, not tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
open_settingsA
Open a system settings panel via ACTION deep link.
Supported panels: home, about_phone, wifi, bluetooth, display, sound, apps, storage, security, location, battery, network, date, accessibility, language, developer.
| Name | Required | Description | Default |
|---|---|---|---|
| panel | No | panel name (e.g. "about_phone" for 设置→关于手机). | home |
| device | No | ADB serial; omit with a single connected device. | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry behavioral disclosure. It mentions the deep-link mechanism but does not elaborate on potential side effects, device/session requirements, or failure modes. For a non-destructive navigation action, this is adequate but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: one main sentence and a list of supported panels. It is front-loaded with the primary purpose and contains no filler. Every element contributes meaningful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 params, no annotations, but an output schema exists), the description covers the essential purpose and scope. It lacks behavior descriptions for session handling or errors, but these are not critical for a straightforward settings opener. The output schema likely documents return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67% (panel and device have descriptions, session_id does not). The description adds a list of valid panel values, which is helpful for the panel parameter. However, it does not provide additional meaning for device or session_id, leaving the undocumented session_id poorly contextualized. This partially compensates but not fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Open a system settings panel via ACTION deep link.' It identifies the specific verb (open), resource (system settings panel), and mechanism. The list of supported panels adds specificity and distinguishes it from the sibling open_app tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by enumerating supported panels, which serves as a whitelist of valid targets. It doesn't explicitly mention alternatives or exclusions, but the scope is clear enough. A sibling comparison is absent, but for a simple settings opener, the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
press_keyA
Press a system key: back, home, recent, menu, enter, power, volume_up, volume_down, clear.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | one of the supported key names (e.g. "back"). | |
| device | No | ADB serial; omit with a single connected device. | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention potential side effects (e.g., power toggling screen, volume changes), device connection requirements, or error handling for invalid keys. The list of valid keys helps but omits any system-level impact or prerequisites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently communicates the core purpose and enumerates valid inputs. No filler words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 3-parameter tool, the description is minimally viable but leaves gaps: session_id is unexplained, and the lack of annotations increases the need for explicit behavioral context (e.g., side effects, multi-device handling). The output schema may mitigate return-value ambiguity, but the description could still offer more operational guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the 'key' parameter by enumerating all accepted values, which goes beyond the schema's single example. However, schema coverage is partial (67%): 'session_id' is left undocumented by both schema and description, and 'device' is only described in the schema. The description does not fully compensate for the missing parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (press) and the resource (system key), enumerating the specific supported keys. This unambiguously differentiates it from siblings like tap, swipe, and type_text, which operate on screen elements or text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for system-level key presses, but does not explicitly contrast with alternatives (e.g., using tap for UI buttons) or define when not to use it. No exclusions or prerequisites are mentioned, leaving the decision to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reset_sessionA
Reset the session: clear per-device snapshot caches and the execution budget counters (spec execution-budget).
Call this after an EXECUTION_BUDGET_EXCEEDED error to continue, or whenever element ids from get_screen have gone stale (a fresh observer is created on the next screen access).
Returns the reset budget state {action_count, elapsed_seconds, max_actions, max_seconds, exceeded}.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does well. It discloses that snapshot caches and budget counters are cleared, that a fresh observer is created on next screen access, and what the return value looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: it states the purpose first, then usage triggers, then return value. Every sentence earns its place with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is highly complete given the simple optional parameter and available output schema. However, the missing explanation of session_id is a small gap that prevents a perfect completeness score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not explain the single parameter `session_id`. Since schema description coverage is 0%, the description should compensate, but it omits any information about how session_id affects the reset operation or what omitting it means.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool resets the session and specifies exactly what is cleared (per-device snapshot caches and execution budget counters). It is a specific verb+resource formulation that distinguishes this tool from the sibling device-interaction tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit usage triggers are provided: call after EXECUTION_BUDGET_EXCEEDED or when element ids from get_screen are stale. This gives concrete, actionable guidance on when to invoke the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scroll_pageB
Scroll exactly one viewport page (native one-page scroll).
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| direction | Yes | "up" | "down" | "left" | "right" — *content* direction ("down" reveals the content below; the finger swipes up). | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It adds only the phrase 'native one-page scroll,' which hints at the scrolling mechanism but does not disclose edge-case behavior (e.g., what happens if the page is not scrollable), whether the scroll waits for completion, or any side effects. The behavior beyond the core action is largely unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that immediately conveys the tool's core function. There is no filler or redundant information, making it optimally concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple and has an output schema, so return values are covered. However, the description omits usage guidance relative to sibling tools and leaves the 'session_id' parameter undocumented in both schema and description. For a straightforward scroll gesture, the description is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes the 'direction' and 'device' parameters with meaningful context (e.g., content direction vs. finger swipe direction). The description adds the concept of 'exactly one viewport page,' which helps interpret the effect of 'direction.' However, it does not clarify the 'session_id' parameter, and the description alone adds minimal parameter-level detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Scroll exactly one viewport page' with the qualifier 'native one-page scroll.' This distinguishes it from siblings like swipe (which can be arbitrary) and scroll_until (which scrolls until a condition), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as swipe, scroll_until, or smart_scroll. It does not mention preconditions, exclusions, or scenarios where this tool is preferred, leaving the agent to guess based on the name and behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scroll_untilA
Scroll until a semantic target appears (or the scroll reaches its end).
Uses one-viewport-page scrolling and stops as soon as the container boundary is reached (no useless swipes at the end).
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc"|"id", "value": str} — the element to find. | |
| direction | No | "down" (default) | "up" | "left" | "right" — *content* direction ("down" searches content below the fold). | down |
| session_id | No | ||
| max_scrolls | No | scroll attempts before giving up (default 10). |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable details: scrolling is one-viewport-page based, and it stops at the container boundary to avoid useless swipes. It does not mention behavior when max_scrolls is reached, but the stopping condition (scroll ends) is partially disclosed, making this a solid 4.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first defines purpose, the second adds a relevant behavioral detail. It is front-loaded, contains no filler, and every word contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and 80% parameter coverage, the description adequately covers the core purpose and stopping behavior. It does not explain return values (not needed) or explicitly mention max_scrolls interaction, but the 'or the scroll reaches its end' phrase covers the natural termination condition. A slightly more explicit note about what happens when the target is not found would elevate it, but it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80%, so most parameters are already documented clearly. The description adds the concept of 'semantic target' which subtly enriches the target parameter by indicating it searches by text/desc/id, but this is marginal. It does not meaningfully compensate for the less-documented device/session parameters, so it stays at the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Scroll until a semantic target appears' with a clear verb and resource, and adds a distinguishing behavioral detail ('one-viewport-page scrolling') that separates it from generic scroll_page or swipe tools. It also clearly conveys the stopping condition (target found or scroll ends), making the tool's purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need to scroll to bring a semantic target into view. It provides context but does not explicitly name alternatives or exclusion criteria. Since it clearly states the primary use case without exclusions, it earns a 4 rather than a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smart_scrollA
Scroll and AGGREGATE every matching element across the screens passed.
Unlike scroll_until (first hit), smart_scroll visits each screen in the scroll budget and returns ALL matches with their positions. Reuses the one-viewport-page scrolling + end detection; on tree-failed screens the OCR layer is aggregated for free.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc"|"id", "value": str} — the element to find. | |
| direction | No | "down" (default) | "up" | "left" | "right" — content direction. | down |
| session_id | No | ||
| max_scrolls | No | scroll budget before giving up (default 10). |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool aggregates over screens, returns positions, reuses scrolling/end detection, and falls back to OCR on tree-failed screens. This is rich behavioral detail, though it does not mention mutation safety, return format specifics, or rate limits—but given the context, the key behaviors are well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action and differentiated from a sibling. Every clause adds value—the comparison, the budget behavior, the reuse of scrolling mechanics, and the OCR fallback. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (aggregation, scroll budget, OCR fallback) and the presence of an output schema, the description is sufficiently complete. It explains the core behavior, how it differs from the closest sibling, and a key edge case (tree-failed screens). The 5 parameters are mostly described in the schema, and the description ties them to the overall workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80% (4 of 5 params described), so the baseline is 3. The description adds value by linking 'scroll budget' to max_scrolls and explaining the aggregation semantics. It also clarifies the target and direction indirectly, though it doesn't add new syntax beyond the schema. The extra context about budget and OCR makes it slightly above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'Scroll and AGGREGATE' with a specific resource ('every matching element across the screens passed'), and immediately distinguishes itself from sibling tool scroll_until by contrasting 'first hit' vs. 'ALL matches with their positions'. This makes the tool's purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Unlike scroll_until (first hit), smart_scroll visits each screen in the scroll budget and returns ALL matches', providing a clear when-to-use vs. alternative. It also mentions reuse of 'one-viewport-page scrolling + end detection' and OCR fallback, giving implementation context that helps decide when to choose this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
swipeA
Swipe the screen by direction (up/down/left/right).
Returns verification evidence {executed, screen_changed, changed_elements[], screen_hash}.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| distance | No | "short" (default) or "long". | short |
| direction | Yes | "up" | "down" | "left" | "right". | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the return evidence structure ({executed, screen_changed, changed_elements[], screen_hash}), which adds useful behavioral context. However, with no annotations, it does not cover side effects, failure conditions, or device requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no wasted words. It front-loads the action and immediately specifies the return evidence, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple swipe tool, the description is largely complete: it mentions the return output and the schema covers most parameters. However, it does not differentiate from overlapping siblings like scroll_page or smart_scroll, and session_id remains undocumented, leaving some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75% (direction, distance, and device have descriptions; session_id does not). The description adds no parameter-level detail beyond what the schema already provides, so it does not compensate for the missing session_id semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Swipe the screen') and the specific scope (direction: up/down/left/right). This distinguishes it from siblings like tap or scroll_page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use swipe over alternatives like scroll_page or smart_scroll. The description only states what it does, not the appropriate context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tapA
Tap a semantic target and verify the result.
Prefer semantic selectors — never compute coordinates yourself:
by=text: exact text label, e.g. {"by": "text", "value": "保存"}
by=desc: content-desc label
by=id: a stable element id from get_screen
by=bounds: last resort, "x,y" or "l,t,r,b" (raw coordinates)
Returns verification evidence {executed, screen_changed, changed_elements[], screen_hash}, or SELECTOR_AMBIGUOUS with candidates[] when several elements match (retry with by=id).
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc"|"id"|"bounds", "value": str} | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses the return shape (verification evidence fields), the SELECTOR_AMBIGUOUS case with candidates[], and a retry recommendation. It does not mention side effects or waiting behavior, but the verification result indicates the tool confirms its own action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a one-sentence purpose, a bulleted selector guide, and a return-format note. Each component contributes unique value; no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the main behavior (verification, ambiguity handling), selector semantics, and output shape. The schema covers device and session_id. It omits potential edge cases like timeouts or post-tap settling, but for a tap tool the description is sufficiently complete alongside the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning to the target parameter by explaining each 'by' value (text, desc, id, bounds) with examples and a 'last resort' note. This goes beyond the schema's type-only description. session_id is not covered, but schema coverage is moderate and the main required parameter is well documented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource: 'tap a semantic target and verify the result.' Distinguishes from sibling tools like swipe/type_text by focusing on semantic selectors and verification. The selector hierarchy (text/desc/id/bounds) further clarifies its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on selector choice: 'Prefer semantic selectors — never compute coordinates yourself,' with a hierarchy (text, desc, id, bounds last resort). Also advises retrying with by=id on ambiguity. However, it does not explicitly compare to alternative tools like swipe or type_text, though the distinct purpose makes this less critical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
type_textA
Type text into an input field.
With target, the field is located semantically first (must be an editable field). Without target, the currently focused field is used. The field is cleared before typing.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | the string to input. | |
| device | No | ADB serial; omit with a single connected device. | |
| target | No | optional semantic target for the input field. | |
| session_id | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden and reveals key behaviors: semantic location of the target, the editable-field requirement, and that the field is cleared before typing. It stops short of describing failure modes or edge cases, but the core side effect is explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences deliver the core action, the target/focus decision, and the clearing behavior without filler. The description is front-loaded and every sentence contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with an output schema and no annotations, and the description covers the main operational aspects: what is typed, how the field is selected, and the clearing side effect. It could add failure/error behavior for non-editable fields or a missing target, but it is largely adequate for selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents text, device, and target at 75% coverage. The description adds meaning for target (semantic location, must be editable) and clarifies that the field is cleared, but it does not add value for device or the undocumented session_id parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Type text into an input field,' giving a specific verb and resource that clearly distinguishes it from sibling tools like press_key or tap. It further defines the input mechanism (semantic target or currently focused field) and the clearing behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear conditional usage: use a semantic target when the field is not focused, or rely on the current focus when target is omitted. It also warns that the target must be an editable field, but it does not explicitly contrast with alternative tools like press_key.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verify_elementA
Assert whether a semantic target exists on the current screen.
Structured boolean result — never raises for not-found or a text mismatch, so the model can branch on it directly.
| Name | Required | Description | Default |
|---|---|---|---|
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc"|"id", "value": str} — the element to check. | |
| text_predicate | No | optional substring the element label must contain. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for behavioral disclosure. It clearly states two critical behaviors: it never raises on not-found or text mismatch, and it returns a structured boolean result. This is valuable and goes beyond the input schema by clarifying error handling and outcome type.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise: two sentences that front-load the primary purpose and immediately add a crucial behavioral note. Every word earns its place, and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values are covered. The description covers the core purpose and the non-raising behavior, which is the key contextual information for an assertion tool. It slightly lacks detail on edge cases like timing or off-screen elements, but for this simple tool it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema fully documents all three parameters. The description itself adds no parameter-specific details, but the baseline score of 3 is appropriate since the schema handles the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb ('Assert') and resource ('semantic target') scoped to the current screen. It also implies differentiation from siblings by emphasizing the boolean, non-raising result, though it does not explicitly name alternative tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: when the model needs to branch on existence or text mismatch without exceptions. It does not explicitly list exclusions or alternative tools, but the guidance is sufficient for selecting this tool over siblings like locate or wait_for.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_forA
Block until a semantic target appears or disappears on screen.
Polls fresh snapshots every interval seconds until the condition
holds or timeout expires (handles loading screens and transient UI).
Never raises on timeout — returns satisfied: false + final hash.
| Name | Required | Description | Default |
|---|---|---|---|
| state | No | "present" (default) waits until it appears; "absent" until it disappears. | present |
| device | No | ADB serial; omit with a single connected device. | |
| target | Yes | {"by": "text"|"desc"|"id", "value": str} — the element to wait for. | |
| timeout | No | max seconds to wait (default 10). | |
| interval | No | seconds between polls (default 0.5). |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so excellently. It discloses blocking behavior, polling interval, timeout handling, and the critical non-raising timeout behavior ('Never raises on timeout — returns satisfied: false + final hash'). This goes beyond the schema by explaining runtime behavior and result semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two compact paragraphs: the first line states the purpose, and the second adds essential behavioral details. Every sentence contributes value—no filler, no repetition of schema fields. It is front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (5 params, nested target object) and the existence of an output schema, the description fully covers the key aspects: what it does, how it behaves (polling, timeout, return), and the edge case of loading screens/transient UI. It is sufficiently complete for an agent to select and invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and each parameter already has a descriptive schema comment. The description adds context about interval/timeout ('handles loading screens and transient UI') but does not explain target structure or device semantics beyond what the schema provides. This aligns with the baseline 3 for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Block until a semantic target appears or disappears on screen.' This clearly distinguishes wait_for from siblings like verify_element (which likely checks current state) and locate (which finds an element), as it emphasizes blocking/polling over time.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: 'Polls fresh snapshots every interval seconds until the condition holds or timeout expires (handles loading screens and transient UI).' This implies when to use the tool (waiting for dynamic UI states) but does not explicitly name alternatives or exclusion cases, which would be needed for a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
v0.4.0- Added
diff_state - Added
locate - Changed
open_app1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Changed
open_settings1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Changed
press_key1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Added
reset_session - Changed
scroll_page1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Changed
scroll_until1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Added
smart_scroll - Changed
swipe1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Changed
tap1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Changed
type_text1 field changed- added
Input schema / properties / session_idAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null +}
- Added
verify_element - Added
wait_for
12 tool updates
v0.1.0- First observed
get_device_info - First observed
get_screen - First observed
get_screen_hash - First observed
list_devices - First observed
open_app - First observed
open_settings - First observed
press_key - First observed
scroll_page - First observed
scroll_until - First observed
swipe - First observed
tap - First observed
type_text
TDQS
Scored across 18 tools
Most tools target distinct actions or resources, but the four scroll variants (swipe, scroll_page, scroll_until, smart_scroll) and multiple verification helpers (wait_for, verify_element, diff_state) occupy adjacent territory. The descriptions do enough to separate them, so only one or two selections could initially be confused.
Names are mostly snake_case imperative verb phrases like get_screen, open_app, and press_key, but a few like wait_for, scroll_until, and smart_scroll break the strict verb_noun pattern. The overall style is still consistent enough that an agent can predict tool names.
At 18 tools, the surface is on the heavy side; several scroll and screen-verification helpers could potentially be consolidated. Each tool has a purpose, but the sheer number makes selection harder and feels borderline for an agent.
Core UI automation is covered well: screen capture, actions, waiting, verification, and navigation. However, there are notable gaps around app lifecycle management (no install/uninstall/stop) and no explicit screenshot capture tool, which can create dead ends for some phone-control workflows.
Maintenance
Related MCP Connectors
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
MCP server connecting AI agents to 100+ apps (Gmail, Slack, Notion, GitHub) via one-click OAuth.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceA Model Context Protocol server that enables AI agents to control and automate Android devices through natural language, supporting actions like app management, UI interactions, and device monitoring.59MIT
- AlicenseNot gradedqualityDmaintenanceA MCP server that enables LLMs to control Android devices via ADB, supporting input, UI hierarchy, device management, and shell commands.14MIT
- AlicenseNot gradedqualityDmaintenanceAn MCP server that provides tools for controlling Android devices using uiautomator2, enabling AI to automate tasks like tapping, swiping, and managing apps.87 PyPI44Apache 2.0
- AlicenseNot gradedqualityCmaintenanceMCP server that enables LLMs to control Android devices via ADB, providing tools for screen interaction and UI inspection.1MIT