Server Quality Checklist

Tool Scores

• get_current_weatherA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses the data source (AROME/INCA/nowcast), the out-of-coverage notice behavior, and that the response indicates which datasets served it. This goes beyond a simple read-only claim and gives the agent a concrete model of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the main purpose. The 'Args' section adds example values without redundant explanation, and every sentence contributes useful information. No filler or vague language is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter read-only tool with an output schema available, the description covers the essential aspects: geographic coverage, data source, parameter format, and error behavior. It does not need to explain return values because the output schema exists, and the description satisfies all other contextual needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only lists 'latitude' and 'longitude' with no descriptions, so the description must add meaning. It does so by specifying 'Decimal latitude' and providing concrete examples (48.2208 for Vienna), plus instructing the agent to geocode city names. This compensates fully for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get current weather conditions for a location,' clearly naming the verb and resource. Since sibling tools cover hourly forecast, storm outlook, and air quality, this tool is distinctly positioned as the current-conditions option.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the geographic limitation ('Austria and the Alpine region only') and instructs the agent to geocode city names to coordinates itself. It does not explicitly reference sibling tools, but the scope and parameter requirements give clear guidance on when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_hourly_forecastA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: the data source (AROME/C-LAEF), forecast horizon (up to ~60 h), hours clamping (1–60), and the start parameter semantics ('begins at/after this instant'). It also warns about out-of-coverage responses. It doesn't mention auth, rate limits, or side effects, but for a read-only forecast tool these are less critical; the provided behavioral details exceed what a title/schema alone would offer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-sentence purpose, a brief context sentence with data source and coverage, a note on out-of-coverage behavior, and a clearly separated Args section. It is concise enough for a 4-parameter tool, and every sentence contributes either purpose, constraints, or parameter semantics. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex with 4 parameters and an output schema. The description covers all necessary aspects: what it does, where it works, parameter details, and edge-case behavior (out-of-coverage). Since an output schema exists, the description doesn't need to explain return values. The coverage is complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain parameters. It does: latitude/longitude with a concrete example, hours with default and clamp range, and start with an ISO 8601 example and behavior ('forecast begins at/after this instant'). Every parameter in the schema is accounted for, and the description adds meaningful constraints and formats beyond the bare type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Get an hour-by-hour weather forecast for a location.' This clearly distinguishes it from sibling tools like get_current_weather (current conditions) and get_storm_outlook (storm alerts). The mention of 'hour-by-hour' and 'forecast' makes the tool's unique scope obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states clear usage context: it serves Austria and the Alpine region only, and explicitly notes that a point outside the grid returns an out-of-coverage notice. It also instructs the agent to geocode city names to coordinates itself, implying a prerequisite. It does not name alternative tools explicitly, but the geographic constraint and hour-by-hour nature create clear usage boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_air_qualityA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and excels: it discloses the geographic coverage limitation, the model-forecast nature, the possibility of an empty response due to stale/incomplete runs, and that an out-of-coverage notice differs from a blank response. This goes well beyond basic behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured: a one-line summary, followed by output specifics, coverage/limitations, edge-case behavior, and parameter notes. Every sentence adds relevant information without redundancy, achieving conciseness through organization rather than brevity for its own sake.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the existence of an output schema (which covers return values), the description covers all necessary context: what it returns, geographic coverage, data source caveats, how to handle edge cases, and parameter preparation. It leaves no obvious gaps for an agent to misuse the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. The Args section provides decimal examples for latitude/longitude (Vienna coordinates) and explicitly states 'Geocode city names to coordinates yourself,' giving meaningful semantic guidance beyond the bare number type in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get air quality for a location: pollutants now and the AQI outlook.' It further details the exact pollutants (NO₂, O₃, PM10, PM2.5) and the European Air Quality Index bands, making it unmistakably distinct from sibling weather/storm tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context and restrictions: 'serves Austria and the Alpine region only,' 'model forecasts, not station measurements,' and instructs users to geocode city names themselves. However, it does not explicitly name alternatives or state when not to use this tool versus its siblings, stopping short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_storm_outlookA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it does so thoroughly. It explains the data source (GeoSphere AROME), the CAPE gating by convective inhibition, the thunderstorm scan horizon and its caveats, the rounding of time windows, and the meaning of timestamps for ongoing thunderstorms.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but every sentence adds unique value. It is structured with a clear opening, a list of returned metrics, data-source context, coverage caveats, and parameter details. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's complexity, the description covers the returned values, edge cases (stale or truncated runs, out-of-coverage notices), and interpretation of results. An output schema is present, but the description goes beyond it to explain behavioral nuances, making it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no descriptions for latitude and longitude (0% coverage), but the description fully compensates by giving decimal examples ('48.2208 for Vienna') and instructing the user to geocode city names to coordinates themselves. This adds clear meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get the severe-weather outlook for a location: gusts and thunderstorms', which names a specific verb (get), a resource (severe-weather outlook), and the content (gusts and thunderstorms). This clearly distinguishes it from sibling tools like get_current_weather and get_hourly_forecast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool, such as the geographic scope (Austria and the Alpine region) and the note that it deliberately reports no severity verdict, leaving judgment to the caller. However, it does not explicitly name alternative tools or state when not to use it, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.