vision_detect
Parse screen-parsing detection JSON into AI-friendly elements with bounding boxes and center points for GUI agent control. Accepts inline or file input and returns window-relative coordinates.
Instructions
Parse screen-parsing detections (OmniParser v2 compatible) into AI-friendly elements with bbox/center. Read-only; no model runs on this host.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "vision_detect"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| score | No | ||
| origin_x | No | window top-left inside the image, X; subtracted to reach window-relative coords | |
| origin_y | No | window top-left inside the image, Y; subtracted to reach window-relative coords | |
| detections | No | inline detection JSON (used when detections_path is absent) | |
| max_results | No | ||
| detections_path | No | path to a detection JSON file |