Server Quality Checklist

Tool Scores

• get_weather_statsC
  2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only states what the tool does at a high level ('Get weather statistics information') without revealing any behavioral traits such as whether it requires authentication, has rate limits, returns historical or aggregated data, or what format the output takes. This leaves critical operational aspects undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise ('Get weather statistics information'), which is efficient but under-specified. While it avoids unnecessary words, it fails to provide enough context to be useful, making it feel more like under-specification than effective brevity. It's front-loaded but lacks substance, so it doesn't fully earn its place as a helpful description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity implied by 'weather statistics' (which could involve historical data, aggregates, or comparisons), the lack of annotations, no output schema, and a vague description, this is incomplete. The description doesn't clarify what 'statistics' entails, how data is returned, or any operational constraints, leaving significant gaps for an agent to understand and use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100% (since there are no parameters to describe). With no parameters, the description doesn't need to add semantic details beyond what the schema provides. The baseline for 0 parameters is 4, as there's no parameter information to compensate for or elaborate upon.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get weather statistics information' is a tautology that essentially restates the tool name 'get_weather_stats' with minimal elaboration. While it indicates the general domain (weather statistics), it lacks specificity about what statistics are retrieved, what time periods are covered, or what geographical scope applies. It doesn't distinguish this tool from its siblings like 'get_current_weather' or 'get_weather_forecast' beyond the vague term 'statistics'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions. With siblings like 'get_current_weather', 'get_weather_alerts', and 'get_weather_forecast', there's no indication of when 'statistics' are appropriate versus 'current', 'alerts', or 'forecast' data, leaving the agent to guess based on the tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_alertsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal insight. It doesn't describe whether this is a read-only operation, potential rate limits, authentication needs, data sources, or what 'weather alert information' entails (e.g., severity levels, expiration times). The agent must infer behavior from the generic 'get' verb.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose, though it could be more informative. The brevity is appropriate but risks under-specification rather than true conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that likely returns structured alert data. It doesn't explain return values (e.g., list of alerts, timestamps, types), error handling, or how optional location filtering affects results. For a tool with potential complexity in weather data, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'location' documented as optional for retrieving all alerts. The description adds no parameter-specific details beyond what the schema provides, such as format examples (e.g., city names, coordinates) or how location filtering works. Baseline 3 is appropriate since the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get weather alert information' restates the tool name with minimal added specificity. While it includes the verb 'get' and resource 'weather alert information', it doesn't distinguish this tool from its siblings (like get_current_weather or get_weather_forecast) beyond the general alert focus. This is borderline tautological with the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention its siblings (e.g., get_current_weather for current conditions, get_weather_forecast for predictions, or search_locations for location lookup), nor does it specify contexts like emergency planning or risk assessment where alerts are most relevant.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_locationsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the action ('search') without disclosing behavioral traits like what 'supported locations' means, whether results are paginated, if authentication is needed, or any rate limits. This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, making it easy to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple but vague purpose, the description is incomplete. It doesn't explain what 'supported locations' are, what the search returns, or how it fits into the server context with sibling weather tools, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'query' documented as 'Search keyword'. The description adds no additional meaning beyond this, as it doesn't elaborate on query syntax, examples, or constraints. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Search supported locations' states a clear verb ('search') and resource ('locations'), but it's vague about what type of locations and lacks specificity about the search scope. It doesn't distinguish from sibling weather tools, which are unrelated but share the same server context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention any context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_weatherC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions nothing about rate limits, authentication needs, data freshness, error conditions, or response format. 'Get current weather information' implies a read-only operation, but this isn't explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - a single sentence that communicates the core functionality without any wasted words. It's front-loaded with the essential information and earns its place efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'current weather information' includes, how recent the data is, potential limitations, or what format the response takes. Given the lack of structured metadata, the description should provide more operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal value beyond the input schema, which already has 100% coverage with a clear parameter description. The description mentions 'for a specified location' which aligns with the schema's 'location' parameter, but provides no additional semantic context about location formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get current weather information') and target resource ('for a specified location'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like get_weather_forecast or get_weather_alerts, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_weather_forecast or get_weather_alerts. It simply states what the tool does without context about appropriate use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_forecastC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe important traits: whether it's read-only or has side effects, authentication requirements, rate limits, error handling, or what the forecast includes (e.g., temperature, precipitation). This leaves significant gaps for a tool that likely queries external data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's appropriately sized for a simple tool and front-loads the core purpose immediately. Every word earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a weather forecast tool. It doesn't explain what the forecast returns (e.g., time periods, metrics), potential limitations (e.g., accuracy, data sources), or error cases (e.g., invalid location). For a tool with external dependencies and likely structured output, this leaves the agent under-informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters ('location' and 'days'). The description adds no additional parameter semantics beyond what's in the schema. It mentions 'for a specified location' which aligns with the 'location' parameter but provides no extra context about format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('weather forecast'), and specifies the target ('for a specified location'). It distinguishes from sibling 'get_current_weather' by focusing on forecast rather than current conditions. However, it doesn't explicitly differentiate from 'get_weather_stats' which might also involve forecast data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'get_weather_forecast' over 'get_current_weather' or 'get_weather_stats', nor does it specify prerequisites or exclusions. The agent must infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.