find
Locate UI elements by name, returning clickable coordinates via accessibility or OCR fallback for desktop automation.
Instructions
Find elements by name. Returns refs with boxes (accessibility) or, if nothing accessible matches, OCR text boxes with a click_at point in screenshot pixels.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ocr | No | Fall back to OCR when accessibility finds nothing | |
| query | Yes | Text to look for: a button label, field name, visible text | |
| scope | No | 'all', 'focused', or app/title substring | all |