Server Quality Checklist

Tool Scores

• get_pricingA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context about pricing being list prices before discounts and that free models are zero. However, it does not disclose limitations such as rate limits, authentication requirements (though 'public' implies none), pagination behavior, or response format, leaving some gaps for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly written sentence that front-loads the core purpose and includes only essential caveats (discounts, free models). Every clause adds value, and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with two optional parameters and no output schema, the description covers the key semantics and caveats. It explains the pricing basis and points to a dashboard for effective rates. It could benefit from describing the return structure, but the tool's simplicity makes this less critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides full descriptions for both parameters (limit and query), achieving 100% coverage. The description adds no parameter-specific details beyond what the schema already documents, so it remains at the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Look up per-token pricing for UnoRouter models from the public pricing endpoint.' It uses a specific verb ('look up') and identifies the resource (pricing for models). This distinguishes it from sibling tools like search_models and chat, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to interpret the results ('Prices are USD list prices per 1M tokens before per-group discounts') and points to an external alternative for effective rates (the account dashboard). It also notes that :free models are always 0. However, it does not explicitly mention when to use this tool versus the sibling tools, though their purposes are distinctly different.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_modelsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden and does well by noting the catalog is 'live', that :free models cost nothing, and that results are usable with specified clients. It does not mention rate limits or error behavior, but for a search tool it provides useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action and resource, and every sentence adds value—covering scope, cost, and return usage. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 optional parameters, no output schema) and solid parameter descriptions, the description adequately covers purpose, return values, and integration points. It could mention pagination but the schema's limit and default are documented, so completeness is high.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all three parameters, so the description adds little beyond what the schema already provides. The mention of ':free' cost complements the free_only parameter, but this is not essential for understanding parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the live UnoRouter model catalog, a specific resource. It also explains what it returns (matching model IDs) and their usability, distinguishing it from a generic search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need model IDs for chat or any OpenAI-compatible client) but does not explicitly compare with sibling tools like get_pricing. It lacks clear 'when not to use' guidance, though the purpose is fairly evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chatA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description takes on the transparency burden. It discloses that :free models cost nothing, have a light per-model rate limit, and that 429 errors can be mitigated by rotating models – useful behavioral context. It does not mention authentication, error handling beyond 429, or streamed responses, but the core behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences: the first states the core function, and the second provides guidance on model selection and rate limits. Every sentence adds value without redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a chat tool with no output schema, the description sufficiently states that it returns the reply. It covers model selection and cost/rate-limit nuances. It omits details about the exact response shape or error conditions beyond 429, but these are inferable for a simple chat invocation. The guidance on siblings adds useful context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 60% (model, prompt, system have descriptions). The description adds meaning for the model parameter by linking it to search_models and explaining the :free suffix, but does not elaborate on prompt, system, max_tokens, or temperature meanings beyond the schema. It partially compensates for the uncovered parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Send a single prompt to any UnoRouter model and return the reply' – a specific verb, resource, and scope. It distinguishes itself from siblings (search_models, get_pricing) by focusing on the chat/completion action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to 'Use search_models to find model IDs', naming the alternative tool. It also provides practical usage guidance about :free models, rate limits, and rotating models on 429 responses, which is valuable for choosing and using the tool correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.