marks
Captures a screenshot of a window with numbered marks over accessible controls and returns a JSON legend mapping each mark to the control's details, enabling precise clicks via mark numbers.
Instructions
Set-of-Marks capture: a screenshot of the window WITH its accessible
controls drawn as numbered red marks, plus a JSON legend mapping each
number to the control's role, name, current value, and exact global
click point. One glance replaces the estimate-zoom-estimate loop for
every control the accessibility tree knows: read the number off the
image and call click_ui(mark=N) (or pointer at the legend's x,y).
window is an address from desktop (default: focused); name filters
the marked controls. Falls back to the plain legend when ImageMagick is
not installed, and to a fall-back-to-vision note when the app exposes
no tree (then use screenshot + zoom).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| window | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |