find_on_screen
Find any on-screen text and get its exact coordinates. Searches XML hierarchy first, then OCR fallback, to verify if buttons, labels, or messages are visible.
Instructions
Find specific text on the screen and return its location. Searches XML elements first (fast), falls back to OCR if not found. Use this to check if a button, label, or message is visible. Returns JSON with {text, x, y, w, h, method} or null if not found.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"device": "<device>",
"text": "<text>"
},
"name": "find_on_screen"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| device | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |