Image Label MCP Server

Outline objects (Segment Anything)

segment_objects

Trace precise object outlines from rough boxes or points with Segment Anything, then save them as polygon annotations for image labeling.

Instructions

Trace precise outlines with Segment Anything and add them as polygon annotations.

Give each object a rough box (or a point on it) in image pixels; the model finds the object's exact edges, so the box only has to contain the object. Prefer this to drawing polygons yourself. model: "sam" for photos and general images; "medsam" for medical images (CT, MRI, X-ray, ultrasound, endoscopy, pathology), which takes boxes only. Up to 20 objects per call. With save=false the outlines are returned without being saved. Each result has the model's quality score (0-1).

Input Schema

Name	Required	Description	Default
`save`	No	Save the outlines as polygon annotations; false only returns them.
`model`	No	'sam' for everyday images; 'medsam' for medical images (CT, MRI, X-ray, ultrasound, endoscopy, pathology), which needs a box for every object.	sam
`objects`	Yes	Objects to outline, each with a label and a rough box (x, y, width, height) or a point on it, in image pixels.
`image_id`	Yes	Id of an image in Image Label: the imageId (a UUID string) returned by list_images or an upload.

Output Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

Changed4 schema fields changedv0.8.1Oct 28, 2026

addedInput schema / properties / image_id / description

Added value: +"Id of an image in Image Label: the imageId (a UUID string) returned by list_images or an upload."

addedInput schema / properties / model / description

Added value: +"'sam' for everyday images; 'medsam' for medical images (CT, MRI, X-ray, ultrasound, endoscopy, pathology), which needs a box for every object."

addedInput schema / properties / objects / description

Added value: +"Objects to outline, each with a label and a rough box (x, y, width, height) or a point on it, in image pixels."

addedInput schema / properties / save / description

Added value: +"Save the outlines as polygon annotations; false only returns them."

Addedv0.3.3Oct 24, 2026

TDQS

A4.8/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the write/safety profile; the description adds the operational context an agent would otherwise learn by failing: a 20-object batch cap, that a loose box suffices, that medsam accepts boxes only, that save=false is a dry run returning outlines unsaved, and that each result carries a 0-1 quality score. No contradiction with readOnlyHint=false / destructiveHint=false.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and the key insight about rough boxes, and the operational limits come next. It loses a little by restating the model enum description almost verbatim from the schema, which is duplicated text rather than new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter mutation tool with an output schema and full annotation coverage, this is complete: batch limit, dry-run mode, model semantics, input units and quality-score presence are all covered. Return-value detail is correctly left to the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns above that by explaining that the box 'only has to contain the object' (removing the most common calling error) and by tying model choice to image domain. It does not add much about color/notes, but those are self-evident from the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Trace precise outlines with Segment Anything and add them as polygon annotations') and explicitly distinguishes itself from the manual alternative in the sibling set ('Prefer this to drawing polygons yourself'). An agent can tell it apart from add_annotations without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives selection criteria for the model parameter ('sam' for photos, 'medsam' for CT/MRI/X-ray/ultrasound/endoscopy/pathology), states the box-only restriction for medsam, and defines the save=false path. It also names the alternative workflow (hand-drawing polygons) and says to prefer this tool, which is real routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.