computer.observe
Observe the current computer state—screen geometry, active window, cursor, OCR text, and accessibility UI elements—to understand the screen before an AI agent clicks, types, or scrolls.
Instructions
Observe the current computer state: screen geometry, active window, cursor, OCR'd on-screen text and accessibility UI elements. Levels: 'minimal' (fast), 'normal' (screenshot+OCR+accessibility), 'full' (adds focused element details). This is the primary tool for understanding the screen before acting.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "computer.observe"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| level | No | normal |