Forecast Job

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=false, and idempotentHint=true, so the safety profile is covered. The description adds output content but discloses no behavior beyond that — no mention of file-reading requirements, that it is a pure computation, or any cost/time characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single dense sentence, but it leads with an ambiguous clause rather than the core action, and the bulk of the text enumerates return values that the output schema already supplies. Size is moderate, but the front-loading is poor.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the return values need not be explained, so the description's output enumeration is largely redundant. Annotations and 100% param coverage carry the rest; the only real gap is the missing clarity on the tool's action and when to prefer it over siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (way, items, thinking, concurrency, promptfoo_results, wrong_answer_cost) is already documented in the schema. The description adds no parameter-level meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The phrase 'The whole job quoted from a promptfoo run of a sample of it' gestures at extrapolating a full job from a sampled run, but 'quoted' is an unclear verb and the purpose is buried under output-column language. It distinguishes itself weakly from siblings like estimate_cost/estimate_time but never states cleanly what action it performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the mention of a promptfoo sample file (with the --filter-sample example) hints that you must have a sample run first, which functions as a de facto precondition. There is no explicit when-to-use, when-not-to-use, or routing to any of the nine sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.