Suite From Traces

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description richly discloses behavior beyond the readOnlyHint annotation: cases expect tool trajectory plus latency/cost within a margin, are marked needs_review, have PII masked unless 'mask' is false, failed runs are skipped with a reason, and nothing is stored. The 'Nothing is stored' claim is consistent with readOnlyHint=true.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is front-loaded in the first clause, with supporting behaviors following compactly. It is dense but each clause carries information, with little wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description needn't explain return values, and it covers the transformation semantics and side-effect profile well. However, for a 10-parameter tool at 20% schema coverage, a few tuning parameters (strategy, include_output) remain unexplained in both schema and description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is low (20%), so the description must compensate. It clarifies 'mask', and implies cost_margin/latency_margin via 'latency/cost within a margin', but leaves 'strategy' (redact vs hash), 'include_output', and the runs payload shape undocumented; salt and session_id are covered only by their schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: converting successful trace runs into a draft regression suite, and details what each generated case contains. The operation is clearly distinct from siblings like lens.suite_check, but no sibling is named or contrasted explicitly, so it falls short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance and no routing to alternatives such as lens.suite_check or lens.compare_runs. The version prerequisite (agenticlens>=0.8.0) is stated, but nothing tells the agent when this tool is the right choice versus its siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.