ghost_find
Locate UI elements by name, role, description, or on-screen text, returning center coordinates, bounding rectangle, and confidence. Works without focusing or raising the window, even when covered or on a hidden desktop.
Instructions
Locate an element (name|role|description|text) and return center (always), rect (has_rect=true for UIA/cache hits), source, confidence, name, escalated (true = a network VLM call was paid). Target window = window= (anchored) or the session anchor; else the foreground. Under the default background policy nothing is focused or raised: the lookup is scoped to the window's own UIA subtree, so it works while the window is covered or lives on a hidden desktop.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | 'instant' (default): local tiers, auto-escalates to VLM on miss. 'deliberate': VLM from first attempt. 'instant_only': no VLM. | |
| name | No | Accessible name (case-insensitive substring) | |
| role | No | Control type: button, edit, checkbox, list, menu, tab, toolbar | |
| text | No | On-screen text for OCR-based location | |
| index | No | Select the nth match (0-based) when several elements share the name/role; name+role AND-combine on this path; returns matches count | |
| window | No | Title substring of the window to search (anchors it). Omit to use the anchor. | |
| description | No | Natural-language description for VLM grounding |