Skip to main content
Glama

pixel_to_world

Read-only

Convert image pixel coordinates to 3D world points by intersecting camera rays with a floor or custom plane, so you can measure positions and sizes from a photo after camera matching.

Instructions

Turn picture pixels into world points. A ray goes from the camera through each pixel and hits the plane z=plane_z (the floor by default) or plane {"point": [x,y,z], "normal": [x,y,z]}. pixels is a list of [x, y]: x to the right, y down, from the top left of a picture of image_size [width, height]. It uses the field of view, the sensor fit, the lens shift and orthographic cameras. A point is null if the ray is parallel to the plane or the plane is behind the camera. Use it to read sizes and positions off a photo after match_camera (for example the width of something that stands on the floor). Heights above the plane cannot be read from one view.

Input Schema

NameRequiredDescriptionDefault
planeNo
cameraYes
pixelsYes
plane_zNo
image_sizeYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.4.0

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and lower the bar, but the description adds real behavioral detail beyond them: the null result when the ray is parallel to or behind the plane, and support for field of view, sensor fit, lens shift and orthographic cameras. It does not describe return container/ordering or rate/cost characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place (mechanism, defaults, coordinate convention, edge cases, limitation), and the purpose is front-loaded in the first sentence. The middle is dense with parentheticals and multi-clause sentences that slow parsing, but no content is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still conveys the return shape (world points, possibly null) and the failure conditions that produce nulls, plus the fundamental single-view limitation. Combined with the safety annotations, nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the full burden and does: it defines plane_z (floor by default), the `plane` object shape with point/normal, the `pixels` convention (list of [x,y], x right, y down, from top-left), and image_size as [width, height]. Only the `camera` string is left to inference, which is self-evident from the sibling set.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb+resource ('turn picture pixels into world points') and immediately specifies the geometric mechanism (ray from camera through each pixel hitting a plane), which is distinct from siblings like place_at_pixel (the inverse) and raycast. An agent can identify this as the 2D-image-to-3D-ground unwrap tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the prerequisite and use case: 'Use it to read sizes and positions off a photo after match_camera', with a concrete example (width of something standing on the floor) and a stated limitation ('Heights above the plane cannot be read from one view'). It does not name sibling alternatives (e.g. raycast/place_at_pixel) for related tasks, so it stops short of full when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.