Server Quality Checklist

Tool Scores

• list_weather_stationsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It mentions data source ('stations are loaded from the Icelandic Meteorological Office (api.vedur.is)'), which adds transparency. However, it does not disclose behavioral details such as whether results are cached, update frequency, rate limits, or that it is a read-only operation. The mention of 'Defaults to all' for limit is implied but not clearly stated as a default behavior. Overall adequate but lacks deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted words. The first sentence tells exactly what the tool does and what it returns. The second sentence gives usage guidance. Every part earns its place. Well front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description should explain the return structure. It lists return fields (IDs, names, types, coordinates) which is helpful. However, it does not explicitly state that the tool returns a list, mention any default ordering, or explain pagination behavior implied by the limit parameter. For a simple two-parameter tool, this is mostly complete but could add a note about sorting or the fact that 'all' is returned when limit is omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%—both parameters (limit and region) already have detailed descriptions including defaults and case-insensitivity. The description restates some of this (e.g., 'Defaults to all' for limit, 'Case-insensitive substring match' for region) but does not add new information beyond the schema. With full coverage, baseline 3 is appropriate; no extra value added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists weather stations in Iceland with specific fields (IDs, names, types, coordinates). It explicitly mentions two use cases: looking up station IDs for sibling tools (get_weather_now, get_weather_forecast) and helping users discover stations near a region. This distinguishes it from siblings and gives specific verb+resource scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: to look up station IDs for other tools or to discover stations near a region. It does not explicitly state when not to use it, but the context from sibling names (all weather-related tools) implies this is purely a lookup tool. The region filter hint ('e.g., 'north', 'capital', 'westfjords') provides additional guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_forecastA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description fully discloses key behaviors: data source (scraped from vedur.is), regional coverage, and fallback to national text forecast with a 'degraded' marker on failure. No destructive or side effects are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph. The first sentence states the primary purpose, and every subsequent sentence adds essential detail (variables, scope, fallback) without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description lists returned variables and mentions the fallback behavior. It is mostly complete but lacks explicit details on output structure or station ID usage, though the schema covers the latter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with each parameter already documented (lang, hours, stations). The tool description adds minimal extra meaning beyond reiterating the fallback language, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves multi-day hourly forecasts for Iceland, specifies the variables (temperature, wind, etc.), and distinguishes it from siblings by highlighting the granularity and regional scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for detailed hourly forecasts but does not explicitly state when to use this tool instead of siblings like get_weather_now or get_weather_text. No exclusion criteria or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_nowA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. Lists return data but lacks disclosures about error handling, rate limits, or behavior on invalid stations. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. First sentence states core purpose and data; second covers parameters and discovery. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description should explain return format. It lists fields and notes 'when available'. Could describe response structure (e.g., per station, time of observation) but sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so baseline 3. Description adds meaning: explains stations accept IDs or names, diacritics optional, default to Reykjavík; lang default is 'en'. This enriches schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'get current weather observations' for a specific location (Iceland) and lists the returned data fields. Distinguishes from siblings by specifying temporal scope ('now') and references list_weather_stations for discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use (get current weather) and hints at alternative (list_weather_stations for station discovery). Could explicitly contrast with siblings like get_weather_forecast for future data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_warningsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description discloses it returns structured alert metadata, mentions active alerts, and optional parameters. Lacks details on rate limits or authentication, but sufficient for a read-only weather tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. First sentence captures purpose and outputs, second provides alternative tool reference.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Two optional parameters, outputs clearly described, sufficient for agent to use correctly. No output schema needed for this level of detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so description adds no extra parameter info beyond schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it gets active severe-weather warnings for Iceland from a specific source, lists returned fields (severity, certainty, urgency, etc.), and distinguishes from sibling tool get_weather_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use this tool vs get_weather_text for narrative warnings, providing clear alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_textA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the source and nature of output (prose, narrative), but does not mention error handling or rate limits. Adequate for a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no fluff. Purpose stated first, followed by usage guidance and alternatives. Efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two enum parameters and no output schema, the description provides sufficient context: source, content type, and distinctions from siblings. Missing details about return format are not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with enums. Description adds useful context (Icelandic versions more detailed, defaults) beyond schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets written/prose weather forecasts from the Icelandic Met Office, distinguishing it from sibling tools that provide structured data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (narrative forecasts, text warnings) and provides specific alternatives for structured observations and warnings (get_weather_now, get_weather_warnings).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_earthquakesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description covers default behavior, filtering options, and mutual exclusivity, but could mention data source or latency for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no waste; front-loads the purpose and efficiently conveys key behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description lists return fields and covers all necessary usage context, making it fully adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage, but description adds value by stating defaults and context for region presets (volcanic systems), exceeding the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets recent earthquakes in Iceland with specific fields (magnitude, location, depth, timestamp), distinguishing it from sibling weather tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit defaults (48 hours, exclude unreviewed), explains mutual exclusivity of region and near, and suggests volcanic system monitoring for region presets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.