Step 4: calculate power / plan sample size

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, non-idempotent, and closed-world behavior. The description adds important behavioral context: it is a calculation tool that may require user input (smallest_effect) and offers a conventional fallback. Return values are not described, but for a read-only calculation tool with no output schema, this is a minor gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is densely packed but front-loads the hypothesis-test case and then covers exploration. Every sentence contributes useful information, though a slightly more structured format (e.g., separating modes) could improve readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (7 parameters, no output schema, annotations present), the description covers the two main modes and the most complex parameter (smallest_effect). It omits details on alpha, alternative, and return values, but these are either defaulted in the schema or inferable from the calculation context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 14% (only 'power' is described). The description compensates by explaining smallest_effect (Cohen's d, r, or p₁) and the use_conventional fallback, and it mentions effect_size, n_per_group, and power for exploration. However, alpha and alternative are not explained, leaving notable gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: compute power (hypothesis test) or plan sample size (exploration). It distinguishes two distinct modes of operation, though it does not name sibling tools like run_test or confidence_interval as alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit guidance for two scenarios: in a hypothesis test, ask the user for smallest_effect, and if unknown, pass use_conventional: true; in an exploration, supply two of effect_size, n_per_group, and power. No when-not-to-use guidance is given, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.