Look at a window
lookInspect a Windows app window in the background without bringing it forward: read UI Automation roles, names, values and states, capture covered windows, and OCR image-only interfaces for later actions.
Instructions
See a window in the background, without bringing it forward: UI Automation tree (roles, names, ids, values, states) and, with mode image or both, a native capture that works even when the window is covered (WebP, at most 2000 px; unchanged since the last image sent: not attached again, mode image always attaches it). A window with almost no tree (Tk, canvas, game: fewer than 6 elements) also gets the text of its image read by OCR (ocr.lines: text, point, screenshotId), clicked with point + screenshotId. Selects that window for act; an observation serves one act and expires after 60 s. Returns its observation id: pass it to act (required from protocol 2026-07-28).
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "look"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | text (UI Automation tree, default), image (native capture), both | |
| image | No | With region: screenshotId of the full native image the region is in pixels of (required from protocol 2026-07-28); refused once a newer image of that window replaced it | |
| marks | No | Numbers drawn on the image of mode both, each the index of its tree line: none (default), auto (unnamed controls), all | |
| region | No | [x1, y1, x2, y2] in pixels of the last native image of this window: that area enlarged up to 2x (small text); its own screenshotId takes points | |
| window | No | Window id from windows, or a unique part of its title or app; omitted: the window of the last look |