Server Quality Checklist

Tool Scores

• get_electricity_consumptionA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions cost estimates and 30-minute intervals, but omits details like data freshness, rate limits, authentication needs, or response size. Disclosures are minimal and insufficient for an agent to understand side effects or constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core action and resource. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain return structure. It states 'returns 30-minute interval readings' but lacks details on format, nested fields, or cost estimate representations. For a data retrieval tool, this is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (period_from, period_to) with ISO 8601 formats. The description adds context about half-hourly data and cost estimates, but does not provide additional meaning beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves half-hourly electricity consumption data with cost estimates, using specific verbs ('Get') and resource ('electricity consumption'). It distinguishes well from sibling tools which are about account info and postal areas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not provide explicit guidance on when to use it over alternatives. Since siblings are unrelated, the lack of exclusions is less critical, but still no context on prerequisites or best use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description partially transparent: discloses address exclusion for privacy. However, does not mention other behaviors like data freshness, authentication needed, or error scenarios. Could be more thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded purpose. No wasted words; each sentence provides essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, description covers key aspects of what is included and what is excluded. Could mention if it returns current account or multiple accounts, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so schema coverage is 100%. Description adds meaning beyond schema by detailing what data is returned, giving the agent a clear picture of the output content.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Get' and resource 'account information', lists included data fields (contract status, electricity supply points, etc.), and notes excluded data (address for privacy). Clearly distinct from siblings get_electricity_consumption and get_postal_areas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied use when needing account details, but no explicit when-to-use or when-not-to-use compared to alternatives. Lacks guidance on prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_postal_areasA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly identifies this as a read-only operation by stating 'look up' and explicitly mentions 'No authentication required'. It also describes the return values. However, it doesn't disclose potential limitations like data freshness or rate limits, which is acceptable for a simple lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that cover the purpose, return values, and authentication requirement. Every word is meaningful, and the structure is front-loaded with the key action and outcome.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool without an output schema, the description provides all necessary context: what it does, what it returns, and that it requires no authentication. It is fully complete for an AI agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for the 'postcode' parameter with an example. The description adds value by specifying the output (prefecture, city, area), which the schema does not provide. This helps the agent understand what the tool returns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up', the resource 'area information by Japanese postcode', and specifies the returned fields (prefecture, city, area). It is easily distinguishable from sibling tools (get_account_info, get_electricity_consumption) which serve entirely different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implicitly clear due to the tool's specific domain (postal areas) versus siblings (account info, electricity consumption). It mentions 'No authentication required', indicating public availability. No explicit when-not-to-use guidance is needed given the narrow scope.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.