rerun_failed_experiment

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the full behavioral burden and does so well: it discloses that the call costs money, exactly which units get re-billed per mode (only failed Evaluators vs new Sessions' replays plus evaluations), that successful work is reused rather than redone, and how hub-api signals an invalid request. The reference to ``scoring_counts`` / ``run_counts.failed`` tells the agent how to verify eligibility beforehand.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Strong front-loading: the one-line summary and cost warning come before any detail. But there is real waste — the closing ':param experiment_id: The Experiment's id.' and ':returns:' lines duplicate structured fields, and 'Costs money: judges or replays again' is a slightly garbled restatement of points made more precisely in the paragraphs below.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete enough for a costly mutation: cost model, mode semantics, reuse behavior, refusal conditions, and the follow-up tool are all present, and the ':returns:' line compensates for the absent output schema. Minor gaps remain — no mention of required permissions or whether the original experiment's prior results are mutated in place versus versioned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry the parameter meaning, and the body prose does explain operationally what 'evaluations' and 'replays' each re-run. However, the trailing ':param' block only restates the names and enum values already present in the schema, adding no format, length, or resolution detail for experiment_id. The genuinely additive semantics live in the prose, not the parameter block.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource: 'Retries what failed in an Experiment', and immediately qualifies it as a paid operation. The two supported modes (evaluations vs replays) are spelled out, so the agent knows exactly what class of work the tool performs. It stops short of naming the nearest sibling, rescore_experiment, which an agent could plausibly confuse with the 'evaluations' mode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete selection guidance between the two modes and states the preconditions under which the tool refuses ('when nothing failed or while a side is still running'), plus a pointer to get_experiment for progress. It does not contrast the tool with sibling operations such as rescore_experiment or list_experiment_runs, so the agent must infer which tool to reach for in overlapping scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.