LLM Broker

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false and openWorldHint=true, so the safety profile is partly covered. The description adds genuinely new behavioral context: the Authorization header format and that calls are billed like the REST API. It does not disclose rate limits, error behavior, or the shape of the returned completion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four dense sentences, front-loaded with what the tool does before moving to model defaults, routing, and auth. Nothing is redundant, though the routing example and auth note are packed tightly without much separation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nested-object, open-world generation call with no output schema, the description covers model selection, routing, and billing but omits the return format, how `messages` vs `prompt` interact, and any constraints on max_tokens. Functional but with real gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20%, with just `prompt` documented inline, so the description must compensate. It clarifies `model` (default taylor, ids from list_models) and gives a concrete `routing` example, but leaves `messages`, `max_tokens`, and the routing schema itself unelaborated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: sends messages to the broker using OpenAI chat completions. It references list_models as the source of model ids, which implicitly frames the sibling relationship, but never explicitly contrasts the two tools' roles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives useful invocation context (model defaults to taylor, routing accepts criteria per request), but never says when to choose this tool over list_models or what a typical flow looks like. Usage is implied through the parameter hints rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (readOnlyHint=false, openWorldHint=true, destructiveHint=false) by disclosing the asynchronous payment flow, the fact that the API key is shown exactly once, and the polling contract on status_url. These are exactly the non-obvious behavioral traits an agent cannot infer from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: the core action and the checkout_url/status_url workflow come first, with parameter constraints packed into a compact tail. It reads as a run-on of several concerns, but nearly every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must carry the return contract, and it does: checkout_url for the human, status_url to poll, and the one-time API key reveal. Combined with currency enumeration hints, an agent has everything needed to drive the flow end to end.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With only 50% schema coverage, the description compensates by giving concrete allowed values (topup 10/25/50, monthly 50/100/250, USD/EUR/CHF) and guidance to omit category and level to let the server choose. It still leaves 'level' and the category/level interaction only partially explained, so it falls short of fully covering the schema gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Creates a card checkout (Stripe) for a new broker account') and even maps it to the underlying REST endpoint POST /api/v1/broker/accounts. An agent immediately understands this is account provisioning via payment, not a generic account mutation, which distinguishes it from the read-only siblings (chat, get_balance, list_models).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operational context: hand checkout_url to a human, then poll status_url until 200 to retrieve the one-time API key. It does not name alternatives or state when-not to use it, but no sibling tool overlaps this provisioning role, so the omission is minor.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint/openWorldHint, so the bar is lower. The description adds genuinely useful behavioral context: it's a list-price estimate, in the caller's currency, and the actual bill follows the router's chosen offer (broker.cost in the chat result). This warns the agent the estimate can diverge from the real charge.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the primary purpose, then the caveat. No filler. Could be slightly more explicit about when to call it, but the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter estimation tool with no output schema and sparse schema descriptions, the description is adequate but does not fully explain return format or the token parameters. It covers the key pricing caveat (list vs. actual), which is the most important behavioral note.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 25% (only 'currency' is described in the schema). The description clarifies that a model and token counts are inputs, and that pricing is per-currency, but it does not define input_tokens/output_tokens behavior. It partially compensates for the low schema coverage but leaves gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: 'Cost estimate ... for a model and token counts.' Clear what it does. Does not explicitly differentiate from siblings like get_balance, but the purpose is distinct enough that an agent can pick it out from the list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (you estimate cost before/around a chat call) but never says when to use this versus alternatives, nor any prerequisites. No explicit when-not conditions or alternative routing is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuinely useful context beyond them: the credentials the account sits behind (Authorization: Bearer ast_sk_…) and the fact that amounts are denominated in CHF as the book currency, which matters for interpreting results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with no filler, front-loaded on balance and spend. The trailing 'of the account behind Authorization: Bearer ast_sk_….' is slightly awkwardly appended, but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters, no output schema, and read-only annotations, the description's enumeration of returned fields plus the currency and auth context covers what an agent needs. Only the absence of guidance relative to estimate_cost keeps it from being fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, which sets the baseline at 4. Nothing is required to invoke it, and the description's only argument-like content (the auth header) is not a schema parameter, so no schema-level semantics are missing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description enumerates exactly what the tool returns (balance, credit limit, monthly spend, billing mode, payment currency), so an agent knows the resource precisely and can distinguish it from siblings like estimate_cost or get_model. It lacks an explicit verb, but the resource and returned fields are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance. The closest sibling, estimate_cost, also concerns cost, yet the description never states that get_balance reports actual spend while estimate_cost projects future cost, leaving the routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds the default currency (USD) and the per-1M-token pricing basis, but says nothing about what happens for an unknown id or how scores are scaled/measured.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that leads with identity and scope before enumerating returns. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates the returned payload (scores per category, price per 1M tokens, regions, context, capabilities), which is exactly what the agent needs. It stops short of describing error behavior for a bad id or score semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 50% (id has no schema description), so the description must compensate: 'by id' clarifies that id is the model identifier, and '(default USD)' restates the currency default. This covers both parameters, though it adds no format or example for id.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the resource ('one model ... by id') and anchors it to the sibling list_models, so an agent knows this is the single-item counterpart to the list tool. It lacks an explicit verb and does not differentiate from recommend_model or estimate_cost, which also deal with single models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: fetch details for one model when you already have its id. There is no explicit when-to-use, no when-not-to-use, and no routing advice for cases where the caller lacks an id or wants a recommendation (recommend_model exists as a sibling).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered; the description still adds real context beyond them — the default USD currency, the pointer to GET /api/v1/broker/currencies for other currencies, and the deterministic sort rule (score within category, otherwise price). It omits pagination/limit behavior, which is the main remaining gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence that front-loads the payload (models, scores, prices, regions) before the filtering and sorting rules. Every clause carries information, though the parenthetical currency endpoint reference slightly interrupts the flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description must describe the return shape, and it does list the per-record fields (category scores, price per 1M tokens, regions) plus the ordering. It stops short of explaining pagination or whether limit caps the result set, which an agent would want for a listing tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 40% (limit, region and category are bare), and the description compensates by naming the three filter dimensions (category, region, price) and clarifying the currency param's default and how to discover valid codes. It adds marginal value on top of the schema but leaves limit semantics (page size vs. total) unspecified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (list) and resource (models on the broker) and enumerates exactly what each record carries: per-category benchmark scores (0-1, measured by the provider), price per 1M tokens with currency, and regions. The only sibling, chat, is unrelated, so differentiation is trivially satisfied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage ('Filter by category, region and price') and explains the ordering rule, but never states when to reach for this tool versus not, nor any prerequisite conditions. With a single unrelated sibling there is little to disambiguate, so implied guidance is adequate but not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, openWorldHint=false), so the bar is lower. The description adds real behavioral context: the constraint that only measured models are recommended, the meaning of best vs best_value, and the shape of the response (chat_arguments plus a routing object). It does not mention pagination or latency, but for a read-only recommender this is solid.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Compact and front-loaded: the selection logic comes first, then the measured-models constraint, then the return shape. The dense parenthetical and colon-chained clauses are slightly terse but every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so (chat_arguments, routing object). But for a six-parameter tool with only 33% schema coverage, the parameter surface is left largely unexplained, so an agent cannot fully reason about filtering inputs like region, min_score, or capabilities.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the description does not explain region, min_score, capabilities, or max_price_per_mtok at all. It gestures at 'category' and 'output price', which loosely map to two params, but it leaves the majority of the six parameters undocumented in both schema and prose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (recommends a model from the catalog) and defines the selection logic (best = highest score in category, best_value = score per output price, plus alternatives). This is clearly distinguishable from siblings like list_models and get_model, which enumerate rather than rank-and-select.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the ranking semantics, so an agent can infer it is for choosing a model for a task. However, the description never states when to prefer this over list_models, get_model, or chat, nor does it name those alternatives. No explicit when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false. The description adds real value beyond them by clarifying the consequence: the outcome is aggregated into public per-model stats (failures_30d / outcomes_30d) and that reporting carries 'No refund, no price change'. What remains undisclosed is whether an outcome can be revised or overridden after submission.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action, then supporting details about provenance and effect. Dense but each clause (request_id source, aggregate visibility, no refund) carries information; minor cleanup possible but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-param, no-output-schema reporting tool, the description covers provenance of the key input and the downstream effect, which is the main thing an agent must know. The unexplained 'reason' parameter and the lack of any note on whether a report can be corrected are the only gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry the load. It substantially clarifies request_id by naming its source, and the outcome enum is self-explanatory in the schema, but the optional 'reason' parameter is never mentioned or characterized. That leaves one of three parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource ('Tell the broker if the answer to a request worked') and names the exact artifact it acts on (a request). It is clearly distinguishable from siblings like chat or list_models, which are mentioned only to explain downstream effects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It tells the agent where the required request_id comes from ('broker.receipt of every chat answer') and what the submission affects ('Outcomes appear per model in list_models'), which is strong context. It stops short of stating when NOT to report or how failures should be judged, so it is clear rather than exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.