observe
Look at the phone yourself: a fresh screenshot, the app on screen and (with_text) every labelled or tappable element with a number. Act on an element by its number (act element=N); coordinates, when needed, are in this image's pixels.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"device": "<device>"
},
"name": "observe"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| device | Yes | Device from devices: its name, its label from the dashboard (e.g. "Pixel 7 #3") or its number | |
| numbers | No | draw the element numbers on the screenshot | |
| with_text | No | also list the elements (numbered, centres in image pixels) |