which_metrics_matter

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description bears full burden. It explains the statistical concept (FDR control) and provides an example, but does not describe output format, assumptions, or edge cases (e.g., required dependencies like did_it_change evidence values).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with an engaging tone. The first sentence is a question but quickly leads to explanation. No filler, every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and sibling tools, the description misses key context: the evidence parameter expects output from a sibling (did_it_change), and the output is not described. The tool's niche is clear but integration details are lacking.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds no extra detail about parameters beyond what the schema provides (e.g., alpha default, evidence structure).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool controls false discovery rate across many metrics, contrasting with per-metric thresholds. It includes a concrete example illustrating the problem. However, it does not explicitly differentiate from sibling tools beyond the conceptual niche.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when multiple metrics are tested and false discovery control is needed, but lacks explicit guidance on when to use versus alternatives (e.g., when to use ab_test or score_forecasts).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.