Find Element by Description (AI) [Pro]
find_elementLocate a UI element on a mobile device using a natural language description. Returns coordinates, type, and confidence score. For example, 'the login button' or 'email input field'. Uses AI vision.
Instructions
[Pro] Uses AI vision to find a specific UI element by natural language description. Returns the element's coordinates, type, and confidence score. Use this when you need to locate a specific button, field, or other UI element. Example queries: 'the login button', 'email input field', 'the red error message'.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"device_id": "<device_id>",
"query": "<query>"
},
"name": "find_element"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| device_id | Yes | Device serial ID | |
| query | Yes | Natural language description of the element to find, e.g. 'the blue Submit button' or 'email input field' |