Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=false, so the agent knows this is a safe, repeatable, closed-world read. The description adds that the output includes a pick and per-model break-even token share, which is useful context, but discloses nothing about auth, rate limits, or computation constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single sentence with no wasted padding, and the core action is front-loaded. However, the phrasing 'for a kind of work' and the trailing 'with the break-even share of answer tokens for each against the pick' is syntactically tangled and slows comprehension for an agent scanning tool definitions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-format explanation is unnecessary, and all six parameters are schema-documented. The remaining gap is routing: for a catalog with nine closely related model/cost tools, the description never positions this against its siblings, leaving the agent to guess when comparison is warranted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all six parameters, including the pay enum semantics and the answer_tokens override. The description only glancingly maps to parameters (plan use, break-even share of answer tokens); it adds no naming, format, or default guidance beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb-ish action (two to six models side by side) plus the resource (models for a kind of work) and enumerates the outputs: rating, prices, cost per task or plan use, a pick recommendation, and break-even answer-token share. That is specific enough to separate it from pure pricing tools like estimate_cost, though the sibling differentiation is implicit rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance. With siblings like estimate_cost, find_models, and suggest_for_project in the same catalog, the description never says when comparing multiple models is the right call versus costing or discovering a single one.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/openWorld=false/idempotent, so the safety profile is covered. The description adds genuinely useful behavior beyond that: results use the cheapest listing rather than a specific price, are reported across three time horizons, and when cache tokens are supplied it also returns the no-cache cost and the savings. This is real added context, not a restatement of annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler. The core output shape is front-loaded before the conditional cache behavior, and every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and all six parameters are fully documented in the schema. The description adds the caching nuance and the 'cheapest listing' basis, leaving only cross-tool routing unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description goes further by explaining the conditional effect of cached_tokens and cache_write_tokens on the output (cost without cache plus savings) and by implying the role of calls_per_day through the per-day/per-30-day horizons.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states precisely what is computed: the cost of a workload on each model at its cheapest listing, broken into per-call, per-day and per-30-day figures. The 'cheapest listing' qualifier and the token-cache clause make the scope unambiguous. It does not, however, explicitly differentiate itself from siblings such as estimate_time, price_history or compare_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the framing ('what a workload costs'), so an agent can infer this is the cost-estimation counterpart to estimate_time. There is no explicit when-to-use statement, no prerequisites, and no routing to or away from alternatives like price_history or compare_models.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, closed-world), so the bar is lower. The description nonetheless adds real behavioral context: the estimate is learned from the caller's own timed runs via log_run, and it explicitly discloses the empty-case behavior ('Says so when there are no runs yet'). That empty-state disclosure is meaningful information not present in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core output shape and finishing with the dependency and the empty-case note. The clause 'of that model called that way' is slightly convoluted, but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value documentation is not required. With 100% schema coverage and the description covering the data source, the output shape, and the no-data case, an agent has what it needs; only the explicit when-to-use routing against siblings is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents model, way, tokens, calls, concurrency, and thinking. The description adds no parameter-level detail beyond what the schema provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: it estimates how long one call and the whole job take, and it names the distinguishing output form (a usual time and a slow day). This separates it from estimate_cost and forecast_job by naming 'time' versus cost. It stops short of explicitly contrasting itself with forecast_job, so it is not a full 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the description names log_run as the data source, which tells an agent the estimate depends on prior timed runs. It does not say when to prefer this over estimate_cost, forecast_job, or local_or_cloud, so the agent must infer the routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, closed-world), so the bar is lower. The description adds genuinely useful ordering semantics not in the annotations: results are ranked by rating, each model is pinned to its cheapest listing, and unrated models are appended cheapest-first — behavior an agent needs to interpret the result set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no filler, and the primary content (ranked models meeting limits) leads. The telegraphic style ('with prices per 1M tokens, Arena rating and prompt size') is dense but readable, though the sentence is somewhat clause-stacked.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description does not need to enumerate return fields, and it is adequate for a discovery tool whose six parameters are all optional and mostly documented. The gap is routing: for a tool sitting among nine siblings with overlapping model-selection purposes, the absence of any disambiguation leaves the agent to guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 83%, so the schema already documents task, maker, needs, min_context and max_output_price; only 'limit' lacks a description. The description adds only the loose phrase 'meet the limits', which does not clarify filtering semantics beyond what the schema fields already state, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description makes clear it returns a ranked list of models for a given kind of work, filtered by limits, with prices, Arena rating and prompt size — a distinct scope from siblings like compare_models or price_history. It never names an alternative to differentiate against, and the verb is implied rather than stated, but the resource and output shape are unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance and no routing to alternatives, despite nine siblings in the family, several of which (compare_models, suggest_for_project, not_yet_priced) overlap conceptually. The agent must infer from the name alone when this tool is the right call versus a comparison or estimation tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=false, and idempotentHint=true, so the safety profile is covered. The description adds output content but discloses no behavior beyond that — no mention of file-reading requirements, that it is a pure computation, or any cost/time characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single dense sentence, but it leads with an ambiguous clause rather than the core action, and the bulk of the text enumerates return values that the output schema already supplies. Size is moderate, but the front-loading is poor.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the return values need not be explained, so the description's output enumeration is largely redundant. Annotations and 100% param coverage carry the rest; the only real gap is the missing clarity on the tool's action and when to prefer it over siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (way, items, thinking, concurrency, promptfoo_results, wrong_answer_cost) is already documented in the schema. The description adds no parameter-level meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The phrase 'The whole job quoted from a promptfoo run of a sample of it' gestures at extrapolating a full job from a sampled run, but 'quoted' is an unclear verb and the purpose is buried under output-column language. It distinguishes itself weakly from siblings like estimate_cost/estimate_time but never states cleanly what action it performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the mention of a promptfoo sample file (with the --filter-sample example) hints that you must have a sample run first, which functions as a de facto precondition. There is no explicit when-to-use, when-not-to-use, or routing to any of the nine sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, closed-world), so the description is free to add provenance detail: local speed comes from your own timed runs logged via log_run, electricity from EIA or a supplied price, hardware price is yours. That is useful behavioral context about data dependencies. Minor tension: it cites EIA as a source while openWorldHint=false, though this may be a bundled dataset and is not a clear contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler; the core comparison is front-loaded. The second sentence packs three source notes efficiently, though it is slightly dense and could be split for readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and all 13 parameters are documented in the schema. The description supplies enough context (comparison scope, data provenance) for an agent to invoke this correctly, leaving only sibling routing unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% across 13 parameters, so the schema already carries parameter meaning (including Ollama naming and the local_way enum). The description adds only marginal parameter context, such as the dependency of local speed on logged runs. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete comparison: per-call cloud cost versus buying hardware and running an open model locally, and names the outputs (monthly cost each way, break-even month, whether the computer keeps up). It is clearly distinguishable from generic siblings like estimate_cost, though it never names compare_models or estimate_cost as alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied – the agent can infer this is for local-vs-cloud decisions. The mention of log_run is framed as a data source for speed, not as an alternative tool or a precondition for calling this one. No explicit when-to-use vs compare_models/estimate_cost guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare it is a non-read-only, non-destructive, closed-world write. The description adds genuinely new behavioral context: the write is to a local file on this machine, nothing is transmitted externally, and the stored data is later consumed by estimate_time. It does not cover idempotency or duplicate-run behavior, which the false idempotentHint implies matters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loading the action and its effect, with the privacy guarantee last. Every clause carries information and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full parameter documentation, the description only needs to establish purpose, destination, and privacy, which it does. Minor omissions remain around duplicate handling and whether timing at least one of seconds/promptfoo_results must be supplied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with each of the 8 parameters documented, including the way/model/tokens/promptfoo_results fields, so the baseline is 3. The description only echoes the promptfoo results-file mode, adding no syntax, format, or default-behavior detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb and resource: persist a timed run (or every result of a promptfoo results file) to a local file. It also names the consumer sibling (estimate_time), which helps orient the agent, though it does not contrast itself against other siblings such as estimate_cost or compare_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this after a model call, or when you have a promptfoo results file, to feed the timing estimator. There is no explicit when-not guidance or named alternative for capturing run data differently, so the agent must infer the trigger conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, and closed-world behavior, lowering the disclosure burden. The description adds useful domain context about comparing Arena ratings against LiteLLM's catalogue and estimating pricing delay, but it does not cover operational details such as data freshness or update cadence.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense sentence with no obvious filler, front-loading the core resource definition. However, it reads as a noun phrase rather than a clear verb-led statement, which slightly reduces immediate comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be detailed, and the schema fully covers the sole parameter. The description sufficiently explains the tool's analytical purpose, though it leaves 'recently' and the exact aggregation of pricing delay unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one optional parameter with 100% schema description coverage, so the schema fully documents the maker filter. The description's phrase 'by maker' echoes the parameter's purpose without adding syntax or behavior beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource (recent Arena-rated models missing from LiteLLM pricing) and the computed metric (how soon pricing usually appears). It does not explicitly distinguish this from sibling tools such as find_models or price_history, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states what the tool returns but gives no explicit guidance on when to use it versus alternatives. It implies a use case through its purpose, but there are no when-to-use, when-not-to-use, or sibling-tool comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/closed-world, so the description's job is content disclosure and it does that well: it reveals that results mix observed changes with probabilistic 90-day and 1-year forecasts for unchanged listings, which is behavior an agent could not infer from the schema. It does not mention pagination or result limits, but the read-only profile is already covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is one dense sentence but front-loads the primary payload (changes since 2023) before the forecast clause and the date clause. No filler or marketing language, though the sentence is long enough that the '90 days and year' forecast clause reads as a run-on.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't enumerate return fields, and it still conveys the two modes (history vs. dated snapshot) plus the forecast component. The main omission is guidance on choosing this tool versus siblings, which is a usage gap rather than a completeness-of-behavior gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters, including the YYYY-MM-DD format for date and the listing-name form for model. The description only restates the date behavior ('each listing's prices that day') without adding format or edge-case meaning beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource (a model's listing changes since 2023 plus forward-looking change/retirement probabilities) and what a date adds, so the agent knows this is a historical-plus-forecast lookup rather than a search or cost tool. It never names a sibling to contrast against, so differentiation from find_models or not_yet_priced is left to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use statement or condition selecting this tool over alternatives. The only conditional phrasing ('With a date...') explains the effect of a parameter, not when an agent should reach for this tool versus its nine siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=false, and idempotentHint=true, so the safety profile is covered. The description adds useful semantic context by defining what the three outputs mean (best value = cheapest statistically indistinguishable from top), but discloses nothing about auth, rate limits, or freshness of the underlying ratings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that covers the core action, but the trailing clause ('the best value (the cheapest whose rating the votes cannot tell apart from the top), the top rated and the cheapest rated') is dense and awkwardly nested, requiring re-reading to parse the three tiers.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, annotations covering the safety profile, and 100% schema coverage, the description needn't explain return values or parameter formats. It adequately conveys the tool's purpose and the reasoning behind its recommendations, leaving only usage routing unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so pay, answer, and prompt are already documented with enums, defaults, and format hints. The description adds no parameter-level detail beyond what the schema provides, which is the baseline 3 case.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (suggests a model and effort level) applied to a specific input (a prompt), and clarifies it guesses the kind of work first. It distinguishes itself from siblings like find_models and compare_models implicitly by being a recommendation rather than a lookup, though it never names them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not guidance is given, and no alternatives are named despite a rich sibling set (find_models, compare_models, estimate_cost). The agent must infer that this is the right tool when it wants a recommendation rather than a listing or cost estimate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.