observe
Read the current screen to get UIA targets and a text summary without moving the mouse or keyboard. Optionally include a JPEG for visual judgment.
Instructions
Read the current screen: UIA targets (numbered ids + native coords) and a text summary. Does not move mouse/keyboard. Default image=false (no JPEG) for speed; set image=true only when you must judge pixels visually (then max_width≈960, quality≈60). If changed=false, JPEG is omitted even when requested — do not re-analyze; wait or act differently. If dirty is null, assume changed. Prefer send_actions for multi-step UI; observe after meaningful steps, not after every click. Target ids are valid only until the next UI change.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mark | No | Overlay numbered targets on JPEG when image=true (default true; ignored if image=false) | |
| image | No | Include JPEG (default false). True only to visually read/judge the screen. | |
| quality | No | JPEG quality 40–100 when image=true (default 60) | |
| max_width | No | JPEG max width when image=true (default 960). Clicks always use native coords, not JPEG pixels. | |
| session_id | Yes | Active session id from start_session |