Server Quality Checklist

Tool Scores

• weather_searchC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It doesn't disclose behavioral traits such as whether this is a read-only operation, if it requires authentication, rate limits, or what the output format might be (e.g., list of locations with IDs/names). The description is minimal and lacks essential context for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with a single sentence 'Search for locations matching query', which is front-loaded and wastes no words. However, it might be overly terse given the lack of context, but it earns points for being direct and structured efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and sibling tools present, the description is incomplete. It doesn't explain how this tool fits into the weather context (e.g., is it a prerequisite for other tools?), what the return values are, or any error conditions. For a tool with potential complexity in a weather server, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with parameter 'q' described as 'Location query'. The description adds no additional meaning beyond this, as it essentially restates the schema. With high schema coverage, the baseline is 3, and the description doesn't compensate with extra details like query format examples or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Search for locations matching query' states a clear action (search) and target (locations), but it's vague about what kind of locations (weather-related?) and doesn't distinguish from siblings like 'weather_current' or 'weather_forecast'. It provides basic purpose but lacks specificity and differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. alternatives like 'weather_current' or 'weather_forecast'. The description implies it's for searching locations, but doesn't specify if this is for finding locations to then get weather data, or if it's a standalone search. No explicit when/when-not or alternative usage is mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• weather_currentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool's function but does not cover important aspects such as rate limits, authentication needs, error handling, or response format. This leaves significant gaps in understanding how the tool behaves beyond its basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of a single sentence that directly states the tool's purpose. There is no wasted language, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not address behavioral traits, response format, or usage context, which are crucial for a tool with parameters and no structured output information. This leaves the agent with insufficient information to fully understand the tool's operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('q' for location query and 'aqi' for air quality data). The description does not add any additional meaning beyond what the schema provides, so it meets the baseline score of 3 for adequate but not enhanced parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('current weather for a location'), making it easy to understand what it does. However, it does not explicitly differentiate itself from sibling tools like 'weather_forecast' or 'weather_search', which might offer similar weather-related functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'weather_forecast' or 'weather_search'. It lacks context about use cases, exclusions, or prerequisites, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• weather_forecastC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks critical behavioral details such as rate limits, authentication requirements, error handling, or response format. For a tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality ('Get weather forecast') and includes key constraints (1-14 days, for a location) without any wasted words. It's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral aspects like rate limits or auth, and while the input schema is well-documented, the description fails to compensate for missing context about what the forecast returns (e.g., temperature, precipitation) or how errors are handled.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema already fully documents both parameters (q and days). The description adds minimal value beyond the schema by implying the tool uses location queries and a day range, but doesn't provide additional syntax, format details, or usage examples. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('weather forecast'), and includes the time range (1-14 days) and target ('for a location'). However, it doesn't explicitly differentiate from its sibling tools (weather_current and weather_search), which would require mentioning it provides future predictions rather than current conditions or search functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (weather_current and weather_search). It doesn't mention alternatives, exclusions, or specific contexts for use, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.