Server Quality Checklist

Tool Scores

• get_forecastC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Get weather forecast for a location' but does not describe return format, units, timezone, data granularity, or potential error conditions. This goes little beyond restating the tool's name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the purpose sentence. However, the 'Args' block duplicates the input schema, which adds clutter without value. It is concise in word count but not every line earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool appears simple (two numeric parameters) and has an output schema, but the description still lacks essential context: what the forecast covers (hourly? daily?), the response structure, and how it differs from sibling tools. An agent cannot fully infer usage or expectations from this minimal text.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description coverage (0%), so the description must compensate. However, the 'Args' section merely repeats the names and one-line definitions already present in the schema, adding no units, ranges, examples, or semantic context for latitude and longitude.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb+resource ('Get weather forecast') and identifies the location parameters. However, it does not specify the forecast's time frame (e.g., daily, current) or distinguish it from the sibling tool 'get_hourly_forecast', so it lacks full differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the siblings 'get_alerts' or 'get_hourly_forecast'. There is no mention of scenarios, exclusions, or alternative tools, leaving the selection decision entirely to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_hourly_forecastB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only implies a read-only operation via 'Get'. It does not disclose forecast duration, units, timezone, or other behavioral traits, though it is not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the primary verb. The Args list is somewhat redundant with the input schema but keeps the information organized and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool, the description provides basic invocation details, but it omits the forecast period, units, or how it differs from get_forecast. The presence of an output schema reduces some burden, but the usage context remains thin.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by listing latitude and longitude, but only with generic restatements of the parameter titles. It adds no range, unit, or format details beyond 'of the location'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves hourly weather forecast for a location, using a specific verb and resource. The word 'hourly' helps distinguish it from get_forecast and get_alerts, though it does not explicitly name the alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus get_forecast or get_alerts. There are no prerequisites, exclusions, or contextual scenarios provided, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_alertsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. However, it only states the basic action and parameter, without revealing any behavioral traits such as return format, rate limits, or safety profile. This leaves a significant gap in what the agent can infer about the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, with only two sentences and a parameter explanation. Every word is purposeful with no redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the simplicity, the tool has an output schema that presumably covers return values, and the single parameter is fully explained. The description lacks explicit usage guidance and behavioral details, but for its minimal scope it is mostly complete. A brief note on what kind of alerts are returned would improve completeness, but it is not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only a type and title for 'state', so the description's clarification that it is a 'Two-letter US state code (e.g. CA, NY)' adds essential meaning beyond the schema. This fully compensates for the 0% schema description coverage, though it could mention validation or edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Get weather alerts for a US state.' This is unambiguous and naturally distinguishes it from sibling tools like get_forecast and get_hourly_forecast, which focus on forecasts rather than alerts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving alerts but does not explicitly contrast with the forecast siblings. It provides the context of 'US state' but lacks explicit exclusions or alternative tool mentions, so usage guidance is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.