Benchmark the engine's top picks with real test queries

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavioral context beyond annotations: auto-expansion of test queries (5 to 10/15), parallel execution, 15+ live LLM calls, and the fact that user-specified model names are ignored. These details enrich the readOnly/openWorld/idempotent annotations without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-organized: core action first, then mechanics and used cases, then exclusions and cost. Every sentence adds value, with no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers what the tool does, how it works (query generation, parallel runs), when to use (after pick/rank), when not to (user-specified models), and cost implications. An output schema exists, so detailed return-value documentation isn't required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single `purpose` parameter is fully documented in the schema (coverage 100%) with guidance to be concrete. The description reinforces the role ('stated purpose', 'generates representative test queries') but introduces no new semantic detail beyond the schema's parameter description, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource+scope: 'Run a live A/B test against the engine's TOP 3 PICKS for a stated purpose.' It clearly differentiates from siblings by noting the engine selects candidates from the full catalog and contrasts with `compare` for user-specified models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use and when-not-to-use guidance: 'Use AFTER pick or rank when the user wants the engine's own picks stress-tested' and 'DO NOT use this when the user has already named specific candidate models — Use compare instead.' Also mentions cost relative to rank.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.