replay_corpus

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does well: it discloses strict read-only behavior, exact rolling ingestion window semantics, model-name constraints for source='codex', the scan cap of 4x limit, and result reporting of considered vs eligible sessions. Minor gap: it doesn't state whether results are returned incrementally or only when complete, and doesn't detail output shape beyond provenance, but an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact for 9 parameters and uses multiple sentences purposefully. It front-loads the core purpose and read-only nature, then explains key parameters in order. The PRD floors aside is slightly dense but valuable. Not every parameter gets an example, but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex retrieval/replay tool with 9 optional parameters, no annotations, and a rich output-schema, the description is thorough. It covers the selection algorithm, eligibility thresholds, and provenance reporting. The presence of the output schema fills return-shape details, so the description is complete enough for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. The text explains days, source, model_family/project/session_category, limit, min_turns/min_tool_calls, and mechanisms implicitly. It grounds the default floors (5/5) against PRD §6's documented floors (10/10). It omits a detailed explanation of 'mechanisms' and 'project' semantics, but given the filter set is mostly intuitive, the description adds substantial meaning over the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource ('Reproduce the shipped compression mechanisms... report the aggregate real tool-result token reduction'), clearly distinguishing it as a batch replay/evaluation tool over a corpus. It also contrasts with siblings like replay_session and replay_compare by its corpus-wide aggregate focus.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly explains when to use the tool, how the corpus is filtered, and which parameters narrow it. It communicates the relationship between the 'limit' parameter and scan behavior, the default vs PRD floors, and includes exclusions ('why the rest were excluded'). While it doesn't name an alternative tool explicitly, it provides clear and detailed guidance on invocation context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.