Server Quality Checklist

Tool Scores

• models_get_pricingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds value by detailing the return format (perUnit price in rubles) and error behavior (InvalidParams for unknown/inactive model IDs), going beyond what the annotations alone convey. Slightly short of a 5 because it doesn't mention any potential side effects or edge cases beyond errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured. The first sentence delivers the primary purpose, the second provides usage guidance and contrasts with siblings, and the third mentions error behavior. Each sentence earns its place, and the most critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but indicated), the description adequately covers when to use the tool, what it returns (perUnit price in rubles), and how errors are handled. It doesn't need to enumerate return fields since the output schema exists, and the description covers all necessary operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for modelId is already thorough: 'Exact model id from the catalogue, vendor included, e.g. "openai/gpt-6-astra"'. The tool description does not add additional parameter semantics beyond implying that the ID must be valid and active (which is inferred from the error behavior). Since schema coverage is 100%, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'The full price of one model in rubles' and explicitly distinguishes it from siblings by noting that models_list and models_search may return '0.00' for non-token models, while this tool provides the 'perUnit' price. This leaves no ambiguity about what the tool does or how it differs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use this tool: 'Prefer this over the pricing embedded in models_list and models_search whenever the model is not billed per token.' It also mentions error conditions (InvalidParams for unknown/inactive models) and clarifies the output unit, making the choice and invocation straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• balance_getA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (read-only, idempotent), the description adds behavioral context: requests are refused when balance is spent, and the account can go slightly negative. This goes beyond the annotation hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with two clear paragraphs. It avoids unnecessary details and delivers the essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the output representations (balanceKopecks for arithmetic, balanceRub for display) and the behavior regarding balance depletion, making it complete for a simple no-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and schema coverage is 100% (empty). The description does not add parameter-related information, but since there are none, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the current account balance, using the verb 'get' and specifying the resource. It distinguishes itself from siblings by the explicit focus on balance, though not directly comparing to others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use the tool: 'Worth checking before a long or expensive run'. Also explains the behavior of balance going negative, which informs usage decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• models_listA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the annotations by explaining the pagination mechanism: 'Paging: read `nextCursor` from the response and pass it back as `cursor`. When `nextCursor` is absent you have reached the end.' It also provides an estimate of the number of calls needed ('about 11 calls at the default page size'), giving the agent a realistic expectation of the tool's behavior. The annotations already indicate read-only, idempotent, and non-destructive behavior, so no contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise. It starts with the core purpose, then explains when to use it, and finally details the pagination behavior. Each sentence serves a purpose, and there is no redundant or filler content. The structure is logical and easy to follow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to list return fields, but it still mentions the key output attributes (capabilities, context window, price). It covers the pagination behavior, usage context, and the tool's scope. This is sufficient for an agent to understand what the tool does, when to use it, and how it behaves without needing to inspect additional references.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions fully cover both parameters (limit and cursor), including the default limit and the instruction to omit cursor for the first page. The description adds no additional semantic information about the parameters themselves; it only references the default page size in the context of the number of calls. Since the schema coverage is 100%, the baseline is 3, and the description provides no extra value beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to walk the entire catalogue page by page, ordered by model id, returning every active model with its capabilities, context window, and price. It also distinguishes itself from sibling tools like models_search (which filters) and models_get_pricing (which gets specific pricing), so an agent can easily differentiate when to use this tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool: 'Use this when you need the complete catalogue — to cache it, to count models, or to scan for something no filter covers.' It also provides an alternative: 'When you are looking for a model that fits a task, use models_search instead.' This gives clear, actionable guidance on both when to use and when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• models_searchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations (readOnlyHint=true, idempotentHint=true, destructiveHint=false). It explains that filtering is done server-side and returns at most 50 rows, that calling with no arguments returns the first 50 active models, and it gives detailed behavioral insight into price-related filters and sorting (how non-token models are handled). This is rich, non-obvious behavior that an agent needs to know. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about 150 words, front-loaded with the core purpose, then expands on optional filters and the price quirks. Every sentence adds value; there is no filler. The structure is logical and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 optional parameters, sorting, price behavior) and the presence of an output schema (so return format is covered), the description is complete. It explains defaults, edge cases, and the reasoning behind price-related behavior. An agent has everything needed to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful nuance to the tricky parameters: it explains that maxInputPer1MRub restricts results to token-billed models, and that cheap_input/cheap_output sorts push non-token models to the end. This adds value beyond the schema descriptions. It does not add much for the simpler parameters (query, capability, minContext), but those are self-explanatory in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Find models that fit a task' and lists the specific filters (name, capability, price ceiling, context window) and sorting. It also positions itself as the tool to use when choosing a model, distinguishing it from siblings like models_list and models_get_pricing. The verb 'search' is specific and the resource is models, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'This is the tool to reach for when choosing a model', which gives clear usage guidance. It also explains that all filters are optional and how they combine (AND). However, it does not explicitly mention when not to use this tool or name alternative tools, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chat_completeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations indicate a non-read-only, non-idempotent, non-destructive operation, and the description adds important behavioral context: it spends real money per token in rubles, is synchronous, and ignores streaming. This goes beyond the basic annotation flags and warns the agent about cost and execution semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a bit longer than strictly necessary, but every paragraph adds value: parity, streaming limitation, model selection guidance, and cost warning. No redundant filler is present, and the structure is logical.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description states the response shape matches OpenAI chat completions, including tool calls, structured output, and reasoning. It also covers cost, model lookup via sibling tools, and streaming behavior, giving the agent enough context to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and each parameter has a detailed, actionable description: model must match the catalogue exactly, messages support multimodal parts, tools require the tools capability, temperature has a caveat for newer models, and reasoning/web_search/response_format are scoped to model capabilities. The description also frames parameters through the OpenAI parity statement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a chat completion against any model in the Hubris catalogue, and distinguishes it from sibling tools by referencing models_search for model discovery and balance_get for cost. It also clarifies the synchronous nature and full parity with the OpenAI chat completions endpoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use the tool: to reach a model not otherwise reachable, after picking an id via models_search, and for delegating subtasks. It also gives a clear negative constraint by stating streaming is not available and the stream parameter is ignored.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.