ocr_screen
Extract all visible text from a device screen via OCR, solving cases where UI elements render as images or canvas. Returns text with bounding boxes, sorted top-to-bottom.
Instructions
OCR the entire device screen using RapidOCR. Returns all visible text with positions. Use this when UI elements are rendered as images/canvas (e.g., analytics dashboards, games, WebViews) where get_elements() returns no text. Returns JSON array of {text, conf, x, y, w, h} sorted top-to-bottom.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"device": "<device>"
},
"name": "ocr_screen"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| device | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |