replay_diff

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description partially carries the burden. It discloses that the tool computes deltas and file intersections but does not mention whether sessions must be complete, if it's read-only, or any side effects. Annotations would have helped.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that efficiently conveys purpose, behavior, and outputs with no wasted words. It is front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (2 string parameters, no output schema, no nested objects), the description is nearly complete. It explains what the tool does and outputs, though it could mention if output is returned directly or via some other means.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description adds context about what comparison yields (metric deltas and file sets), but it does not provide format or validation beyond session IDs. Baseline 3 is appropriate as schema does the heavy listing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies comparing two sessions with explicit outputs (metric deltas and file touches). It distinguishes itself from siblings like replay_status or replay_checkpoint by clearly stating the comparison and side-by-side analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for comparing two sessions but does not explicitly state when to use this vs. alternatives like replay_insights or replay_stats. No exclusions or prerequisites (e.g., sessions must exist) are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.