fermi

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare non-readOnly, destructive, non-idempotent, open-world, but the description adds the details that actually matter for a money-spending call: it charges the saved card real currency (not credits), priced at $0.50/credit with a pricing endpoint to verify, plus the three failure modes (auth failure, no payment method, declined card). That is substantive context beyond the annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with cost, which is the most decision-relevant fact, and organized into Args/Returns/Raises. Some waste: the $0.50 price is stated twice (lead line and 'Credits are $0.50 each'), and the Returns block restates what the output schema already declares.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation that spends real money, everything an agent needs to call it safely is present: price, prerequisite, minimum quantity, credential source, and error conditions. The output schema exists, so the Returns narrative is redundant but harmless rather than missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry the load and it largely does: 'credits' gains a minimum-1 constraint not present in the schema, and 'api_key' gains provenance (from POST /api/v1/accounts) and an explicit required marker. Minor gap: nothing about upper bounds or behavior of non-integer values, though the JSON type covers that.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Purchase credits using a saved payment method') and immediately names the money model ($0.50 per credit) and the off-session mechanism. The sibling tools (estimate, get_estimate, whoami, feedback) are all read/identity helpers with no overlap, so the agent can route unambiguously.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit prerequisite: 'Requires a prior Stripe Checkout session to have saved a card', which tells the agent when the call will fail. There are no true alternative tools for this action, so the absence of an explicit 'use X instead' clause is not a meaningful gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Far beyond what the annotations cover (readOnlyHint=false, openWorldHint=true): it discloses quota spend, anonymous 3/day tier-1 limit, per-tier credit cost (1/3/10), a forced disclaimer acknowledgement, async behavior with HMAC-signed webhooks, and the exact ToolError causes (bad tier, out of credits, unknown provider, missing disclaimer). No contradiction with annotations — the call is a metered, non-idempotent write-ish operation as declared.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Mostly front-loaded — the cost/quota warning comes first, which is defensible for a metered tool — but the purpose sentence is pushed to the second block rather than leading. There is mild redundancy: the neumann / next_steps.simulate hand-off is explained twice, and the Returns section largely restates the existing output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a 12-parameter, open-world, metered tool: billing behavior, async/webhook path, error modes, auth options (api_key omitted = anonymous) and the downstream hand-off are all covered, so an agent can invoke it correctly on the first attempt.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there are 12 parameters, so the description carries the full burden and does so: each argument gets meaning plus examples (unit 'USD', target_horizon '6 months'), tier semantics, the source_inputs item shape (source_type enum values plus content/structured_data), provider/tier compatibility (tiers 2-3 are openai-only), and the coverage_probability default rationale. Nothing material is left undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and artifact: 'Return a calibrated probability distribution for a natural-language question', with a title confirming it creates an estimate. This is clearly distinguishable from get_estimate (retrieval), whoami (account read) and buy_credits (billing) in the sibling list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete selection guidance: tier 1 is fast sync, tier 2 grounded sync, tier 3 deep research async; callback_url removes the need to poll async tiers; whoami reads the same billing block 'without making an estimate'; and neumann should be used when combining multiple estimates. It lacks an explicit 'do not call this when X' exclusion or a direct pointer to get_estimate for retrieval, which keeps it from a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses the side effect (an email is sent to the team plus a copy to the sender), the account-ID attachment behavior when api_key is supplied, and the exact ToolError failure modes. It also pre-empts confusion about the billing block by explaining that charged_to is "none" here, which no annotation conveys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the cost note and the purpose before the structured Args block, so an agent gets the decision-relevant facts immediately. The Args/Returns/Raises formatting is somewhat verbose and restates parameter names, but each entry adds real constraints rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter write-side tool with an output schema, it covers everything needed to call correctly: required vs optional fields, character limits, side effects, error conditions, and confirmation shape. It even explains the billing block, so no follow-up inference is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the full semantic burden falls on the description, and it delivers: each of the five parameters is explained, including the enum values for feedback_type, the 10-5000 character bound on message (a constraint absent from the schema), and the optionality and effect of sender_name and api_key.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The first line states a specific verb and resource: submit feedback, a feature request, or a bug report to the Fermi team. This is trivially distinguishable from every sibling (buy_credits, estimate, get_estimate, whoami), none of which share the feedback function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It opens with "Cost: free" and clarifies that feedback spends neither quota nor credits, which is exactly the context an agent needs before choosing to call it in a budget-sensitive workflow. There are no competing siblings to route between, so the absence of explicit exclusions is minor, but it never states a when-not-to-use condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly/idempotent/openWorld), it discloses cost behavior ('Polling never spends quota or credits'), the auth-scoping rule (same api_key as the originating call or the estimate reports as not found), and the failure mode (ToolError with a REST-equivalent 404). That is exactly the extra context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded (cost note, then one-line purpose, then usage), and the Args/Returns/Raises layout is scannable. However the Returns section is long and partly redundant with the existing output schema, and the free/no-credits point is made twice.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, routing, auth scoping, error behavior, and per-status response shapes, which is more than enough for a two-parameter poll tool with an output schema. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description carries the full burden and does: `estimate_id` is tied to the prior `estimate` response, and `api_key` is explained as principal scoping with an explicit anonymous-tier option. Both parameters gain semantics the bare schema lacks.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Poll the status/result of a previously requested estimate') and immediately scopes it to tier-3 async estimates, which distinguishes it from the sibling `estimate` tool. An agent can route between the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use ('Use this for tier-3 (async) estimates') and the reason the alternative fails (callback_url is unreachable for chat-UI MCP clients such as Claude.ai, ChatGPT, grok.com). It also names the prerequisite: use the `estimate_id` the `estimate` call returned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the read-only and idempotent annotations, the description adds important behavioral context: the call is free, spends no quota or credits, does not consume the anonymous daily allowance it reports, and works anonymously. It also explains bearer-header authentication behavior for the HTTP transport.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loads the critical cost/free behavior before usage and argument details. It is somewhat verbose, repeating the free/no-consumption point and enumerating return fields even though an output schema already exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, read-only account-check tool, the description covers the identity, cost, quota, anonymous-vs-account behavior, and authentication details an agent needs. Since an output schema exists, the additional return-value explanation is not required, but the definition is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema description coverage is 0%, the description fully explains the single `api_key` parameter: where to get it, that omitting it targets the anonymous caller, and that a bearer header can authenticate without passing it. This compensates for the schema's lack of parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific purpose: report the caller's identity, call cost, and remaining quota. This clearly distinguishes whoami from estimate and other sibling tools by focusing on account/quota state rather than producing a cost estimate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit before/after guidance relative to `estimate`: call before to see whether an estimate will spend anonymous allowance or credits, and after to see the new balance. It does not explicitly cover when not to use it or mention other siblings such as `get_estimate`, but the primary usage context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.