Detect UI elements in a screenshot
riv_ui_detectInspect a UI screenshot or design comp to detect elements, returning a nested tree with geometry, fills, and a numbered overlay PNG for building animated prototypes.
Instructions
Find the rectangles, text runs and images in a UI screenshot or design comp. Returns a nested element tree with rects, corner radii and fill colours, plus a numbered overlay PNG. Coordinates come back in the source resolution but are recovered from a downscaled analysis, so expect a pixel or two, and corner radii within about a pixel. Each element carries renderMode ("vector-panel" | "raster" — how it would be reconstructed; flat fills are test-rendered and turned into crops when more than 2% of their visible pixels would differ, the measured share is returned as renderRisk) and semanticHint ("panel" | "text" | "image" | "line" — what it looks like) as independent fields. Look at the overlay, give each element a role, and pass the result to riv_ui_prototype to get an animated .riv. This is geometry only — it does not know a button from a card. If you have the vector source, pass svgPath instead of imagePath: nothing is estimated, the element tree is the SVG's own group nesting, non-rectangular artwork comes through as "vector-shape" with its real bezier vertices, becomes "vector-text" (editable Rive text — the bundled Inter is substituted unless you pass fonts, and a run whose glyphs are missing is baked as a picture instead, always with a warning), embedded data comes through as its own pixels, and Figma layer names arrive as roleHint.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| fonts | No | Fonts for the SVG's <text>. svgPath only — pass the same list to riv_ui_prototype | |
| minArea | No | Ignore regions smaller than this many pixels (default 576). imagePath only | |
| svgPath | No | Vector source instead of a screenshot (Figma/Illustrator SVG export). Nothing is estimated: rects, fills, corner radii and nesting come straight from the file | |
| figmaUrl | No | A Figma link to one frame (Copy link to selection), fetched as SVG through the Figma REST API. Off unless FIGMA_TOKEN is set in the server's environment; without it this errors and nothing is requested. Everything else works offline | |
| imagePath | No | Screenshot or design comp (PNG/JPEG) | |
| maxElements | No | Keep at most this many elements, largest first (default 120, or 300 for svgPath) | |
| overlayPath | No | Where to write the numbered overlay PNG |