run_replay

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description does useful work: it discloses that simulation happens in memory and that results come back as metrics plus a reusable run_id. It does not say whether artifacts are persisted for plot_run, what happens on invalid filter/scenario, runtime cost, or whether a single sample is noisy beyond the hint that one run is one sample.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action, followed by outputs then the usage caveat. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All 7 parameters are undocumented in the schema and mostly unaddressed in the description, with no output schema and no annotations to fall back on. For a 7-parameter entry-point tool that gates the rest of the workflow, this leaves too much for the agent to guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% across 7 parameters, so the description carries the full burden, yet it only touches scenario, trace, and filter. It says nothing about seed (presumably reproducibility), venue, config_patch, scenario_overrides, or the mutual exclusivity/defaults between scenario and trace.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: run one scenario (in-memory simulation) or a recorded trace file through a filter. It also implicitly scopes this as a single-sample run, which separates it from the aggregate siblings ab_test and compare_filters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'One run is one sample: use ab_test or compare_filters for conclusions', naming the alternatives and the condition that selects them. It also names error_breakdown and plot_run as downstream consumers of run_id. It does not clarify when to pick scenario vs trace input, though that is arguably a parameter-level choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.