WhichModel

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond what annotations provide: it specifies that results include an empty changes array when no changes are found, explains how to distinguish empty results from errors using 'total_changes', and describes the return format (old price, new price, model ID, change timestamp). While annotations cover safety (readOnly, idempotent, non-destructive), the description enhances understanding of the tool's behavior in edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured with zero waste: the first sentence states the core functionality, subsequent sentences add crucial behavioral details (empty results handling, error distinction), and the final sentence provides usage context. Every sentence earns its place, and information is front-loaded appropriately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (3 parameters, no output schema), the description is quite complete: it explains purpose, parameters, return format, edge cases, and usage scenarios. The main gap is the lack of an output schema, but the description compensates by detailing the response structure. It could be slightly more complete by mentioning pagination or rate limits, but overall it's well-rounded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema by mentioning the ISO date format for the 'since' parameter and the optional filtering capability, but doesn't provide additional semantic context about parameter interactions or constraints. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific verb ('Returns') and resource ('all LLM pricing changes') with precise scope ('recorded since a given date, optionally filtered to a specific model or provider'). It distinguishes from siblings like 'get_pricing' (which likely returns current prices) and 'compare_models' (which likely compares models rather than tracking changes over time).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool ('monitor cost drift, detect newly added or deprecated models, or build price-change alerts'), giving practical applications. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, which prevents a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds valuable behavioral context beyond annotations: it explains that unknown model IDs are reported without raising errors ('not_found'), describes the return format ('sorted by value score, with a plain-English recommendation'), and mentions the tool's response structure. No contradictions with annotations exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured in three sentences: the first explains the core functionality, the second details parameters and output, and the third provides usage guidelines. Every sentence adds essential information with zero wasted words, making it easy to parse and understand quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (3 parameters, nested objects) and rich annotations (readOnly, idempotent, non-destructive), the description is complete. It covers purpose, parameters, output behavior, error handling, and usage guidelines. Although there's no output schema, the description adequately explains the return format and recommendations, making it sufficient for agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all parameters. The description adds some context by mentioning 'model IDs (e.g., "anthropic/claude-sonnet-4", "openai/gpt-4.1")' and 'optional volume object for cost estimates,' but this mostly reiterates what the schema already specifies. No additional parameter semantics beyond the schema are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool performs a 'side-by-side comparison of 2–5 specific LLMs' across pricing, quality, capabilities, and projected costs. It specifies the exact verb ('compare') and resource ('models'), and explicitly distinguishes it from the sibling 'recommend_model' tool by stating 'Do not use for open-ended model discovery — use recommend_model instead.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool ('when you already have specific candidates and want a structured diff') and when not to use it ('Do not use for open-ended model discovery — use recommend_model instead'). It names the alternative sibling tool directly, giving clear context for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds valuable context beyond this: it specifies the return format (cost per call, daily/monthly projections, comparison to cheapest alternative), which helps the agent understand what to expect. No contradictions with annotations exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the purpose and efficiently lists all return components. Every word contributes to understanding the tool's function without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (cost estimation with projections), rich annotations (safety profile covered), and no output schema, the description is mostly complete. It clearly states what the tool returns, but could benefit from mentioning assumptions (e.g., based on current pricing) or limitations (e.g., does not account for discounts).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, providing clear documentation for all parameters. The description does not add extra meaning beyond the schema, such as explaining token calculation methods or projection assumptions. Baseline score of 3 is appropriate as the schema carries the full burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Estimate the cost'), target resource ('a specific workload for a given model'), and output details ('cost per call, daily and monthly projections, and a comparison to the cheapest alternative'). It distinguishes from siblings like 'check_price_changes' (monitoring) or 'compare_models' (direct comparison) by focusing on estimation with projections.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for cost estimation of workloads, but does not explicitly state when to use this tool versus alternatives like 'find_cheapest_capable' or 'get_pricing'. No exclusions or prerequisites are mentioned, leaving the agent to infer context from sibling names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe, repeatable query operation. The description adds useful context about the tool's purpose (finding cheapest models meeting constraints) which isn't captured in annotations. However, it doesn't disclose additional behavioral traits like potential rate limits, authentication needs, or what happens when no models meet requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise with just two sentences. The first sentence states the core purpose, and the second provides usage context. Every word earns its place with zero waste or redundancy. The structure is front-loaded with the most important information first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a query tool with comprehensive annotations (readOnly, idempotent, non-destructive) and 100% schema coverage, the description provides adequate context. It explains the tool's purpose and when to use it. The main gap is the lack of output schema, so the description doesn't explain what the tool returns (e.g., list of models, pricing details, or just model names). However, given the tool's relative simplicity and good annotation coverage, this is mostly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, all parameters are well-documented in the schema itself. The description mentions 'specific capability requirements' which aligns with the 'required_capabilities' parameter, and 'hard constraints' which relates to all parameters, but doesn't add significant semantic meaning beyond what the schema already provides. The baseline of 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Find the cheapest models that meet specific capability requirements.' It specifies the verb ('find'), resource ('cheapest models'), and constraint ('meet specific capability requirements'). However, it doesn't explicitly differentiate from sibling tools like 'compare_models' or 'recommend_model' beyond mentioning cost-effectiveness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool: 'Useful when you have hard constraints (e.g. must support tool_calling + vision) and want the most cost-effective option.' This gives practical guidance about the tool's intended use case. However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds valuable context beyond this: it specifies default and maximum limits ('default limit is 20 (max 100)'), ordering ('Results are ordered by value score'), and result content details ('Each result includes input/output prices per MTok, context length, max output tokens, capabilities, quality tier, and value score'). This enhances behavioral understanding without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose in the first sentence, followed by details on filters, results, usage guidelines, and exclusions. Each sentence serves a distinct purpose—explaining functionality, constraints, and usage—with no wasted words. It efficiently conveys necessary information in a structured manner.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (7 parameters, no output schema), the description provides comprehensive context. It explains the tool's purpose, filter combinations, result ordering, default/max limits, result content, and specific use cases. With annotations covering safety and idempotency, and the description filling in behavioral and usage details, it is complete enough for an agent to understand and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning all parameters are well-documented in the input schema. The description adds some semantic context by mentioning that 'Filters can be combined' and listing examples like 'specific model ID, provider name, maximum input price per million tokens, required capabilities (tool_calling, json_output, streaming, vision), and minimum context window,' but this largely reiterates what the schema already provides. Given the high schema coverage, a baseline score of 3 is appropriate as the description adds minimal extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'returns raw pricing and capability data for LLM models matching the supplied filters,' specifying both the verb ('returns') and resource ('pricing and capability data'). It explicitly distinguishes from the sibling 'recommend_model' by stating 'Does not make recommendations — use recommend_model for that,' showing clear differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool: 'Use for programmatic price checks, budget validation, or building custom selection logic.' It also specifies when not to use it: 'Does not make recommendations — use recommend_model for that,' naming an alternative sibling tool. This covers both use cases and exclusions effectively.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds valuable context beyond annotations: it mentions the data source ('OpenRouter'), refresh behavior ('Data is refreshed automatically'), and how to check freshness ('check data_freshness in the response'). It doesn't contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured: first sentence states purpose, second lists inputs, third describes outputs, fourth gives usage context, and fifth provides data source details. Every sentence adds value with zero waste. It's appropriately sized and front-loaded with core functionality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 parameters, nested objects) and lack of output schema, the description does well by explaining what the tool returns ('Returns the recommended model, its cost estimate, a reasoning summary, and ranked alternatives') and data freshness. However, it doesn't detail error cases or response format specifics, leaving some gaps for a recommendation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all parameters. The description mentions parameters generally ('Provide a task type and complexity; optionally add token estimates, a per-call budget cap, and capability requirements') but doesn't add specific semantic details beyond what the schema provides. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get the single best-fit LLM for a task, with pricing and reasoning.' It specifies the verb ('Get'), resource ('best-fit LLM'), and key outputs ('pricing and reasoning'). It distinguishes from siblings by explicitly naming alternatives (get_pricing, compare_models).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this when you need to select a model without knowing which to pick.' It also gives clear exclusions: 'Do not use for raw price lookups (use get_pricing) or for comparing specific known models (use compare_models).' This directly addresses when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.