Server Quality Checklist

Tool Scores

• get_price_estimateA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant context beyond the annotations: 'no state changes and no external calls,' 'computed locally from server pricing config,' 'free, no payment, no authentication required,' and 'repeated calls with identical inputs return identical results.' These details clarify side effects, network behavior, and prerequisites, which annotations alone do not convey. No contradiction with the readOnly/idempotent/destructive hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three dense, well-structured sentences with no filler: what it does, safety/behavior, and invocation guidance. It front-loads purpose and money implications before diving into behavioral details, and every clause adds operational value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is an output schema, the description need not detail return values. It covers the required input contracts (model vs mode, messages, max_tokens), the behavioral/read-only path, the cost/auth implications, and points to related tools (chat_completions, list_models). Nothing an agent needs to call it correctly on HTTP is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each property described, so the baseline is already strong. The description goes further by defining mode values ('auto = cheapest model fitting the context, eco = cheapest available, premium = best model') and by clarifying the relationship between model and mode: 'one of two required — if both are sent, model wins.' This resolution semantic is critical and not fully captured by the schema's anyOf.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Estimate the USDC cost of a chat completion request before paying.' It clearly scopes what the tool does and implicitly distinguishes it from chat_completions (the paid execution tool) and list_models (the model-listing tool), reducing ambiguity without even checking sibling schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use this tool: 'Use this tool to check the exact price for a given model/mode, messages, and max_tokens before calling the paid chat_completions tool.' It also gives usage constraints: one of model/mode required, mutually exclusive, and 'if both are sent, model wins.' This tells an agent exactly when and with which arguments to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnly, idempotent, and non-destructive behavior, and the description adds substantial context beyond that: no authentication or payment required, arguments are ignored, data comes from local config, repeated calls return identical results, and there is no filtering or pagination. This gives an agent a complete behavioral picture with no surprises.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence adds distinct value: scope, authentication/read-only/idempotency, parameter handling, absence of filtering, recommended usage, API equivalence, and exclusions. Although it is a longer description, it is tightly packed with actionable information and remains highly scannable because each sentence addresses a separate concern.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, a trivial input schema, and a true output schema, the description fully addresses everything an agent needs to know: cost model, authentication, side effects, behavior with no arguments, and clear routing versus sibling tools. There are no meaningful gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the description expands on this by explicitly stating that the input schema is an empty object and that any arguments passed are ignored. This is more informative than simply inheriting the schema. A 0-parameter baseline is already strong, and the extra clarity about ignored arguments earns a slightly higher score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'List available models and their prices.' It clearly distinguishes itself from siblings by stating it does not generate text (chat_completions) and does not estimate request costs (get_price_estimate). The purpose is unambiguous and fully captures the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use this tool: 'Use this tool to inspect models and prices before calling the paid chat_completions tool.' It also explicitly says when not to use it, naming the alternatives for text generation and cost estimation. This leaves no ambiguity about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chat_completionsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false and carry no safety signals, so the description does the heavy lifting. It clearly discloses the required payment flow (x402, first call returns 402, retry with _meta['x402/payment']), non-streaming responses, and max_tokens clamp/default behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but each sentence earns its place. Payment requirement and routing rules are front-loaded before the 'Use this tool…' usage note. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a paid mutation-style tool with no safety annotations, the description covers all prerequisites, parameter selection, defaults, and which sibling to use instead. Combined with an output schema existing, nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema coverage is 100%, the description adds meaningful semantics: explains auto/eco/premium routing, mutual exclusivity with model, model wins if both, and max_tokens default/clamp. This goes beyond the schema's basic type/enum definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb and resource: 'Send a chat completion request to any supported model.' It also frames it as the text-generation tool, opposite to the free inspection tool list_models, so the agent can immediately distinguish it from its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use this for text generation and to use free list_models first to inspect models and prices. It also explains the mode-vs-model selection, which is essential for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.