understand_screen
Analyze screen content to identify applications, text, and actionable insights. Optionally provide a prompt for targeted analysis like diagnosing errors or explaining dashboards.
Instructions
Understand what's on screen — like Google Lens for your desktop. Returns structured analysis: what app, what content, what's happening, actionable insights. Optionally provide a prompt for focused analysis: 'what error is this?' or 'explain this dashboard'.
Args: prompt: Optional custom prompt for focused analysis (default: "")
Returns: JSON string with understanding result, image, OCR text, and metadata
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |