Rank LLMs

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish this as a safe, read-only, idempotent operation (readOnlyHint, openWorldHint, idempotentHint, destructiveHint=false), so the bar for additional behavioral disclosure is lower. The description adds value by describing the output structure ('weights, scores, and plain-English rationale per pick') and the intent to compare, which goes beyond what annotations capture. No contradiction exists, and the lack of side-effect discussion is acceptable given the read-only nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two sentences, front-loaded with the primary action and followed by a clear use-case statement. Every word earns its place; there is no fluff or repetition of schema details. The structure makes it easy for an agent to quickly grasp the tool's purpose and when to invoke it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has only 4 parameters (1 required), a rich schema, and an output schema (indicated by context). The description covers the essential context: what the tool does, what it returns, and when to use it. It does not explicitly differentiate from 'compare' or 'benchmark' siblings, but the action 'rank' and the mention of 'shortlist with weights, scores, and rationale' sufficiently convey its role. This completeness is strong but not exhaustive given the sibling ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% – every parameter (top_n, primary, purpose, capabilities) has a detailed description with examples and behavioral semantics (e.g., 'primary' switches to lexicographic ordering; 'capabilities' filters models). The main description does not add parameter-specific meaning beyond what the schema already provides, so the baseline of 3 applies. It neither compensates nor hinders, matching the schema's thoroughness.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's core function: 'Rank LLMs for a stated purpose.' It distinguishes itself from sibling tools by noting the use case is 'to see and compare alternatives, not just one answer,' which contrasts with a pick-one tool like 'pick.' This makes the purpose specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the appropriate context: 'Use when the user wants to see and compare alternatives, not just one answer.' While it doesn't name alternative tools or specify when-not-to-use, this clear use-case guidance provides enough direction for an agent to select this tool over siblings. The absence of explicit exclusions or named alternatives keeps it just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.