replay_compare

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden, and it delivers: it declares 'Strictly read-only', explains the fetch-once-and-reuse behavior, discloses the default config-set behavior when configs is omitted, and exposes the surprising min_turns/min_tool_calls default of 5/5 versus the PRD's 10/10. It even notes where this appears in output ('reported in each config row's provenance').

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence carries distinct information: purpose and example, read-only guarantee, configs format and default, corpus selection reference, and floor overrides with provenance note. The description is dense but well-organized, front-loading the core purpose and behavior before parameter details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the essential behavior, all parameter groups, defaults, and output provenance for a tool with 9 optional parameters and a non-trivial configs structure. The only notable gap is that corpus-selection parameters are documented by reference to replay_corpus rather than described inline, and there is no mention of potential errors or limit semantics, though an output schema exists to fill in return details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does for the complex configs parameter with a concrete example and default behavior, and for min_turns/min_tool_calls with floor values and PRD context. The corpus-selection parameters (days/source/model_family/project/session_category/limit) are grouped and delegated to 'same as replay_corpus' rather than individually explained, which is helpful but not fully self-contained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: 'Replay the same session corpus under multiple mechanism configurations in one call'. It also names the exact purpose and outcome ('so bundles stay attributable') and gives a concrete comparison example. This clearly distinguishes it from siblings like replay_corpus and replay_session by its multi-config focus.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes clear context: you use this tool to compare multiple mechanism configurations in a single read-only call, with an explicit example. It references replay_corpus for corpus-selection semantics, implying a shared/different usage pattern, but it never explicitly states when to use replay_compare instead of replay_corpus or replay_session, nor gives a when-not-to-use exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.