find_elements
Parse the screen and return only elements whose label contains a query, optionally filtered by type, so you can locate and click the right UI target immediately.
Instructions
Parse the screen and return only the elements whose label contains query (case-insensitive
substring; empty query = all), optionally filtered by type (button, link, text, icon, …).
One call instead of parse_screen + scanning 300 elements. The ids are valid for
click_element(id) right away (this IS the most recent parse).
Returns {"ok", "instance", "vision", "query", "type", "count": total matches, "elements":
[{id, type, label, interactive, center}, …] (first limit, 1–100), "truncated"}. "count": 0
is an honest answer (the text is not on screen, or OCR did not read it — screenshot to check).
Fails (isError) like parse_screen; in vision mode 'none' it fails with the fix (no grounding
backend: screenshot and reason yourself).
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"query": "<query>"
},
"name": "find_elements"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| type | No | ||
| limit | No | ||
| query | Yes | ||
| instance | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |