Server Quality Checklist

Tool Scores

• get_weather_by_city_nameB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true (safe read) and openWorldHint=true (supports flexible queries), so the description doesn't need to repeat these. It adds value by specifying 'common Japanese cities', which clarifies scope beyond what annotations provide, but lacks details on rate limits, error handling, or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get weather forecast') without unnecessary words. Every part of the sentence contributes to clarifying the tool's function, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, 100% schema coverage, annotations provided), the description is adequate but has gaps. It lacks output schema details (e.g., forecast format) and doesn't fully address sibling tool differentiation, which could leave the agent uncertain in complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'cityName' fully documented in the schema (including enum values and description). The description adds marginal context by emphasizing 'common Japanese cities', but doesn't provide additional syntax or format details beyond what the schema already covers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get weather forecast') and target ('common Japanese cities by name'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'get_weather_forecast' or 'get_available_city_ids', leaving some ambiguity about when to choose this tool over alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance, mentioning 'common Japanese cities by name' which implies usage for those cities, but it doesn't specify when to use this tool versus siblings like 'get_weather_forecast' or 'get_available_city_ids'. No explicit alternatives, exclusions, or contextual rules are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_available_city_idsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, indicating this is a safe read operation with a closed set of results. The description adds value by specifying the geographic scope ('Japanese cities'), which isn't captured in annotations. However, it doesn't provide additional behavioral context like response format, pagination, or rate limits. No contradiction exists with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently communicates the core purpose without any wasted words. It's appropriately sized for a simple tool with no parameters, and the information is front-loaded with the essential action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with annotations covering safety and scope, the description provides adequate context about what resource is returned. However, without an output schema, it doesn't specify the format of the returned list (e.g., array of strings/numbers), which could be helpful. The description is complete enough for basic understanding but lacks detail about the output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema has 100% description coverage (though empty). The description appropriately doesn't discuss parameters since none exist, which is correct for this case. No additional parameter semantics are needed, so this meets the baseline expectation for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('list of available city IDs for Japanese cities'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get_weather_by_city_name' or 'get_weather_forecast', but the resource specificity (city IDs vs weather data) provides implicit distinction. The description avoids tautology by not just restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention that this tool might be used to obtain valid city IDs before calling weather-related sibling tools, nor does it specify any prerequisites or exclusions. The usage context is implied by the resource type but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_forecastA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the agent knows this is a safe read operation with open-world assumptions. The description adds useful context about the geographic scope ('Japanese city') but doesn't provide additional behavioral details like rate limits, error conditions, or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the essential information without any wasted words. It's appropriately sized for a simple tool with one parameter and good annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation with good annotations (readOnlyHint, openWorldHint) and 100% schema coverage, the description provides adequate context. However, without an output schema, it doesn't describe what the forecast response contains, which would be helpful for agent planning.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the cityId parameter fully documented in the schema. The description adds minimal value beyond what's already in the schema - it mentions 'using city ID' but doesn't provide additional syntax, format, or usage details. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get weather forecast'), the resource ('for a Japanese city'), and the method ('using city ID'). It distinguishes from sibling tools by specifying the city ID parameter approach rather than city name or available IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('using city ID'), which implicitly suggests alternatives like the sibling tool 'get_weather_by_city_name'. However, it doesn't explicitly state when NOT to use this tool or provide explicit comparison guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.