Server Quality Checklist

Tool Scores

• get_forecastA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It explains output includes highs/lows and weather description but omits details like input validation (e.g., invalid city handling), rate limits, authorization requirements, or whether it is a read-only operation. This is insufficient for a tool without annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and contains no redundant information. Every sentence is functional and contributes to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and full schema coverage, the description covers the main functionality. However, it lacks details on return format (e.g., temperature units, structure) and error handling for invalid cities. This is adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds value by noting that city names can be in Chinese or English, but does not elaborate on the 'days' parameter beyond what the schema provides (range 1-3, default 3). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: querying a city's weather forecast for the next few days, including daily high/low temperatures and weather description. It also notes support for Chinese and English city names, distinguishing it from sibling tools like get_current_weather.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly provide guidance on when to use this tool versus alternatives (get_air_quality, get_current_weather). Usage context is implied by the name and description but lacks explicit differentiation or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_air_qualityA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description partially compensates by noting no API key required and listing pollutants. However, it omits other behaviors like rate limits, data freshness, error handling, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences in Chinese that front-load the main purpose. No filler or redundancy; every sentence adds useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description adequately states what is returned (AQI and specific pollutants) but lacks details on the response structure or data format, leaving some uncertainty.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of the city parameter with description. The description adds value by specifying language support (Chinese/English) and providing examples (e.g., Beijing, Shanghai, Tokyo), going beyond basic schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool queries air quality index (AQI) for a city, listing major pollutants (PM2.5, PM10, etc.). It is specific and distinct from sibling tools (weather, forecast), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs siblings (e.g., get_current_weather, get_forecast). It mentions support for Chinese/English names and no API key, but lacks explicit when/when-not context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_weatherA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the output fields (temperature, wind, humidity, etc.) and that no API key is needed. It does not explicitly state it is read-only, but the nature of a weather query implies it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and data points, followed by constraints. No wasted words; highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is complete: it lists all returned data fields, supports both languages, and confirms no API key. No significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter is fully described in the schema. The description adds examples of city names and language support, but these are nearly identical to the schema's description. Thus, minimal added value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool queries real-time weather for a city, listing specific data points (temperature, wind, humidity, etc.). It distinguishes from sibling tools (get_air_quality, get_forecast) by focusing on current weather and specific metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for current weather queries, supports both Chinese and English city names, and mentions no API key needed. However, it does not explicitly state when to use this tool over alternatives like get_forecast or get_air_quality, relying on context from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.