Test coverage by product

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety and scope are covered. The description adds that it includes an overall rollup and untested tickets per product, giving output context but no behavioral traits like rate limits or error handling. With annotations, this is adequate but not exceptional.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loads the core purpose, and wastes no words. Every sentence adds value: the first explains the aggregation, the second lists output components.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description adequately covers the purpose and output shape (overall rollup, untested tickets). It does not mention any filters like date range or status, but given the tool's likely use case and annotations, this is not a critical gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter 'project' (string) with no description in the input schema (0% coverage). The description mentions 'per product' but does not clarify that 'project' likely refers to a product identifier, nor does it specify valid values or format. The agent must infer meaning, which increases ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool rolls up per product the count of tickets with and without recorded test runs, making testing gaps visible. It specifies the resource (tickets per product) and the action (roll up), distinguishing it from sibling tools like get_test_runs or test_runs_by_suite which focus on individual test runs or suites.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that the tool is for making testing gaps visible on the board, providing clear context for its use. However, it does not explicitly state when not to use it or mention alternative tools for more detailed test run analysis, missing a chance to guide agents away from misuse.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.