Server Quality Checklist

Tool Scores

• get_fire_riskA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool readOnly and idempotent. The description adds transparency by disclosing the error result for out-of-range coordinates or unreachable SMHI API, and by clarifying that the indices are unitless values, not raw weather readings. This goes beyond the safety profile provided by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than the minimal but every sentence contributes: purpose, usage, outputs, error handling, and coverage. Its structure is logical and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description carries the full burden of explaining return values. It enumerates the fire risk class, indices, grass fire risk, dryness class, weather variables, and precipitation sums, plus error behavior and geographical coverage. This is sufficient for an agent to know what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all three parameters with 100% coverage. The description adds one extra semantic: the forest dryness class is only available for the daily period, which is not in the schema's period description. This gives the agent useful parameter-dependent behavior information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a wildfire/forest fire risk forecast, explicitly naming the Canadian Fire Weather Index system and the specific output types. It distinguishes itself from sibling general forecast tools by directing users to use this when the question is specifically about fire danger.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this instead of the general forecast tools when the question is specifically about fire danger.' It also gives a coverage area and notes error conditions, helping the agent decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_daily_summaryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description adds valuable behavioral context: error result (isError) for out-of-range coordinates or API unreachability, geographic coverage, and unit conventions. No contradiction with annotations; the description enriches what the annotations already communicate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (four sentences) and front-loaded with the core purpose. Every sentence adds value: content list, usage guidance, units, error handling, and coverage. No redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description fully covers the output shape (high/low, condition, precipitation, wind), units, error behavior, and geographical scope. It also integrates guidance on when to use alternatives, making it self-contained for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal parameter semantics beyond the schema, only reiterating the days range and default. It does not clarify latitude/longitude beyond what the schema provides, nor does it add format details for coordinates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb ('Get') and a clear resource ('daily weather summary') with a precise scope (next 1-10 days, default 7). It enumerates the content (high/low temp, dominant condition, precipitation total, max wind speed), distinguishing it from forecast siblings by emphasizing the daily aggregation and at-a-glance nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('multi-day outlook at a glance') and names the alternatives to use when hour-level detail is needed (get_hourly_forecast or get_forecast). This gives the agent clear decision criteria and prevents misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_hourly_forecastA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds meaningful operational context: return units, default behavior, error result on out-of-range coordinates or API unavailability, and regional coverage. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with the core purpose, then usage guidance, units, error behavior, and coverage. Every sentence provides distinct value and there is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description clearly explains return content, units, default range, error behavior, and geographic coverage. This is sufficient for an agent to select and invoke the tool correctly and interpret results appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema already fully documents all three parameters with ranges and defaults (100% coverage). The description reinforces the hours default and coordinate range constraints but does not add significant new parameter-level semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Get') and resource ('hour-by-hour weather forecast'), with clear scope (1-48 hours, default 24) and content (temperature, wind, precipitation, conditions). Explicitly distinguishes itself from siblings by naming get_current_weather, get_daily_summary, and get_forecast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives a concrete use case ('will it rain in the next few hours') and explicitly lists when to use alternative tools (get_current_weather for a single now-reading, get_daily_summary for multi-day overview, get_forecast for complete raw dataset). Also provides coverage and error conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_warningsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the annotations: it returns all active warnings tagged by county/area, supports filtering by county and severity, defines the severity ordering (RED > ORANGE > YELLOW > MESSAGE), and discloses the error behavior (returns isError: true if the API is unreachable). This is rich context for a read-only tool with already-safe annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is five sentences long, but every sentence earns its place: main purpose, non-coordinate/filtering model, severity ordering, explicit sibling guidance, error behavior, and geographic coverage. It is front-loaded with the primary action and resource, making it easy to scan, with no filler or redundant statements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description provides sufficient information for correct invocation and interpretation: it explains that results are active warnings tagged by area, how to narrow them, severity levels, error cases, and coverage. Given the tool's moderate complexity and strong annotations, this is complete enough for an agent to select and call the tool appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage with detailed descriptions for all three parameters, including the case-insensitive substring match for county, the language enum, and the severity threshold ordering. The description restates some of this (e.g., 'filter by county name and/or minimum severity') but does not add meaning beyond the schema. Thus the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get currently active official SMHI weather warnings/alerts for Sweden.' It further distinguishes itself from siblings by stating it is 'not coordinate-based' and lists the types of impact-based warnings covered. This makes the tool's purpose unambiguous and clearly differentiated from the forecast and fire-risk tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use this for "is there a storm warning" type questions; use the forecast tools for general future conditions and get_fire_risk for the underlying fire danger index.' It also clarifies the tool is limited to Sweden, which helps agents avoid misuse. This goes beyond implied usage to explicitly state when to use and when to use alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_forecastA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, and non-destructive behavior. The description goes beyond annotations by specifying units, time resolution, geographic coverage, and error behavior (returns isError: true on out-of-range coordinates or API unavailability). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph that efficiently covers scope, resolution, unique fields, alternatives, units, errors, and coverage. While longer than the ideal two-sentence example, every clause adds necessary context and the structure is logical.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex forecast tool with no output schema, the description provides all essential context: time resolution, unit system, error handling, geographic boundaries, and explicit relationships to sibling tools. It gives the agent enough information to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents both parameters with ranges (100% coverage), so the description adds limited but valuable semantics: the valid geographic coverage area (Sweden, Norway, etc.) and unit conventions. This supplements the schema without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full ~10-day weather forecast with every raw SMHI parameter at native resolution. It explicitly distinguishes itself from sibling tools by noting it includes fields not exposed by the others, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to use alternatives: 'Prefer get_current_weather for a single now-reading, get_hourly_forecast for an hour-by-hour view, or get_daily_summary for a multi-day high/low overview' and then states 'use this tool when you need the complete unprocessed dataset.' This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_weatherA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, openWorld, idempotent, non-destructive), the description adds critical behavioral details: it returns an error result (isError: true) for out-of-range coordinates or API unreachability, and clarifies that it returns the nearest forecast time step, not actual real-time measurements. It also specifies units for all returned values. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: definition, usage guidance, and units/error/coverage. It is front-loaded with the purpose, and every sentence adds distinct information without repetition or filler. The structure is efficient and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (2 params, no output schema), the description is comprehensive: it lists returned variables, provides usage context, explains units, alleges error behavior, and states geographic coverage. The annotations further cover safety and idempotency. There is no missing critical information for an agent to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for both parameters with ranges. The description adds value by explaining that out-of-range coordinates will produce an error, which clarifies the enforcement of the schema bounds. It also implies the parameters are coordinates within the covered region, though it doesn't detail formatting beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get current weather conditions' and enumerates the data fields (temperature, wind, humidity, precipitation, description). It explicitly distinguishes this tool from siblings by noting it returns the nearest forecast time step, versus hourly or daily tools. This clearly differentiates it from get_hourly_forecast, get_daily_summary, and get_forecast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides direct usage guidance: 'Use this for a quick "what's the weather like right now" snapshot' and explicitly names alternatives for other time horizons. It also mentions geographical coverage, setting expectations for where the tool works. This is explicit and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_radar_imageA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, openWorld, idempotent), the description adds critical behavioral details: update frequency ('updated roughly every 5 minutes'), return payload (PNG, area, product name, timestamps), and error behavior ('isError: true if the SMHI radar API is unreachable'). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences cover purpose, distinction from siblings, usage caveat, return value, error behavior, and coverage. Every sentence adds new information; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description explains the return format and error conditions. With annotations covering safety/idempotency, the description covers the remaining essentials: update cadence, scope, and sibling differentiation. This is fully sufficient for a parameterless read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain beyond what schema shows. The description explicitly notes 'Takes no parameters', which reinforces the schema. Baseline for 0 params is 4, and the description is consistent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb+resource: 'Get the latest Swedish precipitation radar composite', which clearly identifies the tool's function. It distinguishes from siblings by explicitly stating 'Unlike the forecast tools, this shows observed precipitation happening right now, not a prediction.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: it says to use forecast tools for future weather and this tool for current observations. It also notes coverage is Sweden only, which is key for relevance. This gives the agent clear criteria for deciding between this tool and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_analysisA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint, the description reveals response ordering (hourly, most recent first), specific fields with units, error behavior when coordinates are out of range or API is unreachable, and regional coverage. It also notes data availability nuances such as 'later windows only populated at some hours'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Despite being lengthy, each sentence serves a distinct purpose: definition, use cases, return data, error handling, and coverage. The description is well-structured and front-loaded with the core action, containing no redundant or wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description thoroughly enumerates all returned data fields and units, explains the data source and nature, provides usage context, and describes failure modes. It gives a complete picture of the tool's behavior without requiring external documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying that coordinates must be within the indicated geographical coverage and that out-of-range values trigger an error. It also connects the 'past ~24 hours' phrasing to the hours parameter, supplementing the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves past ~24 hours of gridded meteorological analysis from SMHI's Mesan2gv3 API, explicitly noting it is 'not a forecast'. It differentiates from sibling tools by recommending forecast tools for future conditions, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use this tool: for nowcasting, verifying past forecasts, and historical lookups. It also states to use forecast tools instead for future conditions, providing clear alternatives and excluding misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.