Compare two experiments with the same levels side by side to show metric deltas and whether their 95% CIs separate—clarifying before/after changes with error bars.
Two experiments with the same levels side by side: metric deltas and whether their 95 % CIs separate (the before/after question with error bars).
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | ||
| b | Yes |
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Changes observed during successful MCP inspections.
v0.6.0Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish this as a safe read (readOnlyHint=true, destructiveHint=false). The description adds genuine behavioral context by disclosing what is computed (metric deltas, CI separation), but says nothing about what happens when the two experiments do not share levels or how results are scoped.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the core operation and no filler. The parenthetical is slightly dense but still earns its place by clarifying the comparison's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained here. However, with 0% parameter coverage the definition leaves the two required inputs and the 'same levels' precondition undefined, which is a real gap for an agent trying to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the two parameters are opaque single letters ('a', 'b'). The description lets the agent infer they are experiments, but adds no format, identifier convention, or ordering semantics, so the agent must guess what to pass.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific operation (side-by-side comparison of two experiments) and the concrete output (metric deltas and whether 95% CIs separate). It is reasonably distinguishable from exp_evaluate/exp_report by describing the two-experiment, error-bar framing, though it never names a sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'Two experiments with the same levels' and the parenthetical 'before/after question', which hints at when it applies, but there is no explicit guidance on when to prefer this over exp_evaluate, exp_runs, or the analogous rec_compare/snap_compare.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.