spatial_relations
Measure contact, overlap, gap, and containment between named objects in an image. Get pixel distances, embed depth, and shape profiles to verify physical relationships beyond bounding boxes.
Instructions
Measure how named objects in an image sit relative to one another.
Locates each object, segments it, and reports measurements that are hard to judge by eye: whether two things actually touch, how many pixels apart they are, how much of one lies inside the other and how deeply, plus each object's own elongation, straightness and end-to-end width profile.
This reports geometry, not verdicts — it does not decide what is wrong.
Interpret the numbers against what the scene ought to look like: a hand
and the grip it holds that come back separate with a large gap are not
in contact; a hand overlapping a shield with a_inside_b near 1.0 and a
large embed_depth is buried in the shield face rather than gripping its
rim; an elongated object whose end_symmetry is near 1.0 is equally wide
at both ends, unlike a blade that tapers to a point at one end only.
Useful for checking whether a generated or edited image holds together physically, for verifying that an object is where it should be relative to another, and for any question of contact, containment or clearance that a bounding box cannot answer — boxes overlap whenever one object is simply in front of another.
Takes the single best-scoring match per name, so this assumes one instance of
each named object. Asked for 'red circle'/'blue circle'/'green circle' on a
scene with one of each, the detector returned the same three boxes for every
query — color alone doesn't reliably discriminate same-shaped objects — but the
correctly-matching box scored highest every time, which is what this relies on.
For several instances of one kind of thing, give them distinguishing names, or
use count_objects for a tally instead.
A maximally vague name in objects ("object", "thing", "item", ...) is guarded
the same way detect_objects/count_objects are: a near-full-frame box from a
name with nothing distinctive to point to is dropped rather than treated as a
located match, so it doesn't get reported here as the "best-scoring" box for
that name.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| src | Yes | Local file path or http(s) URL of the image to process. PDFs are also accepted and are rendered one image per page, so tools that return a list return one entry per page. | |
| objects | Yes | Names of the objects to locate and compare, e.g. ['hand', 'sword', 'shield']. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||