compare_runs

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose one real behavioral trait: results are partitioned into improvements and regressions rather than returned as a flat diff. It says nothing about the read-only nature, behavior when runs are incompatible, or failure modes, leaving significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with the core action front-loaded and no filler. Nothing is redundant or padded; the brevity only costs it elsewhere, not here.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema covers return values, but the description leaves the two required parameters undefined in meaning and direction and does not explain the compatibility precondition it itself introduces. For a comparison tool with no annotations, this is under-specified in ways that would cause misordering of baseline and candidate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for two required parameters, so the description must compensate and does not. Critically, the direction of the comparison is never established — which of baseline_id and candidate_id is the reference and which is evaluated relative to it — which is exactly the semantics an agent needs and cannot infer from a bare "Baseline Id" title.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ("Compare ... runs") and specifies the output shape (improvements and regressions exposed separately), which distinguishes it from list_runs/get_run. However, the qualifiers "compatible" and "scored" are never defined, so the agent cannot tell what makes a run eligible.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no stated prerequisites (e.g. that both runs must be scored and share a compatible setup), and no reference to alternatives such as get_run or get_study_comparator. Usage is only implied by the verb "compare".

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.