Server Quality Checklist

Tool Scores

• call_modelA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=false, openWorldHint=true, etc. The description adds behavioral context beyond annotations: it states the return value ('assistant text'), explains authentication behavior and quota limits, and distinguishes paid vs. guest usage. It could more explicitly mention side effects like quota consumption, but overall adds meaningful transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: primary action, differentiation from siblings, and authentication context. No redundant or filler words. Front-loaded with the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters (1 required) and no output schema, the description covers the main function, intended usage, sibling exclusion, return type, and authentication context. Minor gap: it does not specify behavior when daily guest quota is exhausted, but the description is largely complete for decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add any parameter-specific explanations beyond what the input schema descriptions already provide (e.g., optional model default, message format, max_tokens). No extra semantic value for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Send a chat completion to Mirai and return the assistant text,' clearly stating the specific verb ('Send') and resource ('chat completion to Mirai'). It explicitly distinguishes this tool from siblings by stating not to use it for looking up prices or quota, which should be done with list_models or account_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit instructions: 'Use this only to generate a reply. Do not use this to look up prices or remaining quota; use list_models or account_status.' Additionally, it explains authentication context—works without an API key while daily guest quota remains, and with a key uses paid API—providing clear context for when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• account_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, non-destructive behavior. The description adds value by confirming it does not send a chat and explaining conditional behavior based on API key. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and conditions, followed by an explicit exclusion and sibling reference. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no parameters, the description covers the main behaviors and usage conditions adequately. Could optionally mention what 'quota' vs 'balance' means, but not necessary for basic selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description doesn't need to elaborate. It still adds context about the conditional behavior (key vs. no key).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads account status (quota/balance) and distinguishes it from sending a chat. It also mentions a sibling tool for prices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool (depending on API key presence) and explicitly directs to list_models for prices, providing clear guidance versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, and non-destructive. The description adds that it reads live catalog data, includes pricing, and requires no API key. No contradiction with annotations; additional behavioral details enrich the agent's understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place. Front-loaded with the primary action, followed by usage guidance, then clarifying constraints. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description clearly indicates the output content: the Mirai catalog and public USD prices per million tokens. This is sufficient for an agent to understand what will be returned. Annotations cover safety profile.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (schema coverage 100% trivially). Baseline is 4 for 0 parameters. Description does not need to add parameter semantics, and it implicitly conveys the tool's function.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads the live Mirai catalog and public USD prices per million tokens. It differentiates from siblings: account_status (likely account-related) and call_model (sending a chat). The verb 'Read' and resource 'catalog and pricing' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this before choosing a model.' Also clarifies what it does not do ('Does not send a chat') and that no API key is required, guiding the agent when to use this tool versus alternatives like call_model.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.