athena_experiment_design

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It discloses a critical trait by returning DESIGN_ONLY, implying no execution or side effects. However, it does not state whether the tool mutates state, what the design output contains in detail, or how it handles invalid inputs, leaving significant behavioral aspects unspecified.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded with the main verb and directly states the purpose and output type. It contains no redundant or extraneous content, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex with 6 parameters, no output schema, and no descriptions in the input schema. The description only provides a high-level overview and a vague 'returns DESIGN_ONLY' tag, without explaining the return format, parameter relationships, or constraints. This is insufficient for an agent to fully understand what the tool returns and how to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It maps 'hypothesis priors/likelihoods' to the hypotheses array and mentions cost and risk weights, but it omits sample_size and control_fraction entirely and does not explain the structure of the array items (hypotheses and experiments). The description leaves the agent guessing about key parameters required for invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (Rank), the object (binary-outcome experiments), and the criteria (expected information gain under supplied hypothesis priors/likelihoods, cost, risk, feasibility, ethics). It also distinguishes itself from siblings by explicitly returning DESIGN_ONLY, indicating it is for experiment design rather than execution or simulation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by mentioning 'under supplied hypothesis priors/likelihoods' (i.e., the user must provide hypotheses and experiments), but it does not explicitly state when to use this tool versus alternatives or provide exclusions. It does not reference sibling tools like athena_counterfactual_simulate, so the agent must infer the intended use case from the design-only emphasis.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.