browser_vision_inspect
Overlays numbered Set-of-Mark badges on interactive elements and captures the view for visual analysis, so you can click a marked element with browser_click(mark=N).
Instructions
Places numbered Set-of-Mark (SoM) badges on interactive elements and captures the view for visual analysis; the numbers are then used with browser_click(mark=N).
Returns the list of elements with box_2d [ymin, xmin, ymax, xmax] on a [0-1000] scale. The capture is written by default to ~/.nexus/nexus-browser/screenshots/ (full-resolution PNG, path returned in output_path). inline=True attaches it to the response as an image instead: WebP, long side at most 2000 px and 2.56 megapixels, at most 500 KB; data.image gives its size and css_per_image_px to read a position on it, and each marker its image_box [x1, y1, x2, y2] in pixels of that image (the badge numbers are drawn on it). Same view as the last image sent for this tab: no image again ("unchanged"), unless force=True. Do not open a raw PNG capture larger than 500 KB with an image reading tool (rejected with HTTP 400).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| force | No | ||
| inline | No | ||
| tab_id | Yes | Your tab's id (from browser_new_tab, or a tab adopted with browser_adopt_tab). | |
| output_path | No |