ShipReady Metrics

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint, idempotentHint, destructiveHint, openWorldHint), so the bar is lower. The description still adds real behavioral context: it returns sourced excerpts and explicitly is not legal advice nor an assertion that ShipReady supports a framework, which sets correct expectations about output meaning.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the action and resource, followed by the essential output disclaimer. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description wisely characterizes the return value as sourced excerpts and caps expectations with the legal-advice disclaimer. Adequate for a single-question lookup tool, though a hint about how results are scoped would round it out.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

One parameter with 100% schema description coverage ('Compliance question'), so the schema carries the semantics. The description adds nothing about question phrasing, scope, or examples, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Find relevant public ShipReady compliance guides.' This is clearly distinguishable from the only sibling, scan_public_repo, though the description never names or contrasts it explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the resource name (compliance questions about ShipReady), but there is no explicit when-to-use guidance, no statement of prerequisites, and no contrast against scan_public_repo.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and open-world, so the safety profile is covered. The description adds genuinely new behavioral context: results are point-in-time, limited to a score, counts and up to three findings, and are explicitly not a certification or release approval.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, zero filler, with the core action front-loaded and the constraints and disclaimer following in descending order of importance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully discloses the shape of the response (score, counts, up to three findings) and sets expectations via the non-certification disclaimer. A little more on latency, rate limits, or how the score should be interpreted would close the remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage, so the schema already documents it fully. The description adds no URL format or syntax detail beyond the schema, which is the expected baseline for a one-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Scan) plus resource (public GitHub repository) and even names the backing pipeline (/scan). It does not differentiate from the sibling ask_compliance, so it falls short of a 5, but the operation is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The scope exclusion 'no private repos or tenant data' effectively tells the agent when this tool does not apply, which is useful. However, it never says when to prefer this over ask_compliance or any other alternative, leaving routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.