redos_bench

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility for behavioral disclosure. It reveals that the tool times both evil and benign strings, sweeps pump sizes, and produces a verdict naming the engine. However, it does not disclose potential computational cost, timeout behavior, or side effects, leaving significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, with two sentences covering the core action, goal, and outcome. It avoids redundancy but uses somewhat cryptic phrasing ('evil vs benign string') that could be clearer without adding length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values are covered, but the description lacks context about prerequisites (e.g., valid regex pattern), how the tool interacts with sibling tools (gen_benign/gen_evil), and what the verdict actually implies. No annotations exist, so the description is the only context and it is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain parameters. It never mentions 'pattern', 'engine', 'qlimit', or 'timeout_ms' by name, and the reference to 'pump sizes' does not map clearly to any parameter. This leaves the agent guessing about how to set parameters correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verbs ('time', 'sweep', 'classify') and clearly identifies the resource (evil vs benign strings) and the goal (classify growth as linear/polynomial/exponential). It distinguishes itself from sibling tools (gen_benign, gen_evil, analyze) by being the benchmarking/classification tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for classifying ReDoS growth patterns but does not explicitly state when to use this tool versus alternatives like analyze or re2_memory. No excluded scenarios or cross-references to sibling tools are provided, leaving usage context vague.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.