Server Quality Checklist

Tool Scores

• get_air_qualityA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core function but omits details such as data source, response format, rate limits, or behavior on invalid locations. The term 'current' is left undefined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with the purpose stated in one sentence and parameter details in a compact Args block. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description is adequate but minimal. It explains the input but does not describe the output structure (e.g., what pollutants are included, AQI scale, or categories), leaving some gap for agent expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a type for 'location' with 0% description coverage. The description adds meaningful semantic guidance: 'City name, optionally "City,CountryCode"' clarifies the expected format. This compensates well for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get current air quality index (AQI) and pollutant breakdown for a location.' It uses a specific verb ('Get') and names the resource (AQI and pollutants), which distinguishes it from sibling weather tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for air quality queries but does not explicitly mention when to use this tool over alternatives like get_current_weather or get_weather_forecast. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_weatherA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the core function and parameters, but does not describe what specific weather conditions are returned, how errors are handled, or any other behavioral traits. This is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient, with one sentence for purpose and a structured list for arguments. Every sentence adds value, and the format is scannable and free of redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool, the inputs are well covered, but the absence of an output schema leaves the return format undefined. The description does not explain what constitutes 'current weather conditions' (e.g., temperature, humidity), nor does it address error scenarios, making it merely adequate with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description thoroughly explains both parameters beyond the schema: 'location' includes optional country code with example, and 'units' lists the exact values and their temperature scales (Celsius, Fahrenheit, Kelvin). This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action and resource: 'Get current weather conditions for a location.' The term 'current' effectively distinguishes it from siblings like get_weather_forecast (future) and get_air_quality (different data type).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is only implied through the word 'current,' suggesting it is for present conditions rather than forecasts. However, no explicit alternatives or exclusions are mentioned, so the agent is not directly guided on when to use this tool vs. siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_weatherA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It implies a read-only comparison operation but does not disclose potential behaviors like error handling, rate limiting, or what happens if a location is invalid. For a simple read tool, the lack of explicit disclosure is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a one-sentence purpose followed by three lines of parameter explanations. No redundant text, clear structure, and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no output schema or annotations. The description covers purpose and parameters well, but does not explicitly describe the return value format. Given the straightforward nature, this is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by defining 'locations' as a list of city names with an example, and enumerating the allowed values for 'units'. This adds significant meaning beyond the schema's type-only definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Compare current weather across multiple locations side by side.' This clearly distinguishes it from sibling tools like get_current_weather (single location), get_weather_forecast (future), and get_air_quality (different metric).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for comparing multiple locations via 'across multiple locations' but does not explicitly state when to use this tool versus alternatives or mention exclusions. The use case is clear but not contrasted with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_forecastA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It focuses on input parameters and not on behavioral outcomes, such as the response format or error conditions. The 'Get' verb implies a read-only operation with no side effects, but no further behavioral disclosure is given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, with a clear purpose statement followed by a concise Args list. Every sentence contributes value, and the structure is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers parameters well, but it omits any mention of the return structure (e.g., daily high/low, precipitation) or potential limitations. Since no output schema exists, a bit more detail on the forecast content would improve completeness, but the core use case is adequately clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no property descriptions (0% coverage), so the description fully compensates. It adds critical detail: location format as 'City,CountryCode', days range 1-5, and explicit unit values. This goes beyond the schema's type/default information and is essential for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get a multi-day weather forecast for a location,' using a specific verb and resource. The 'multi-day' qualifier clearly distinguishes it from sibling get_current_weather, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for multi-day forecasts but does not explicitly state when to choose this over get_current_weather or compare_weather. No alternatives or exclusion criteria are mentioned, leaving usage context to be inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.