Server Quality Checklist

Tool Scores

• getHourlyWeatherC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool returns data for 'the next 24 hours' and implies a default location, but doesn't cover error handling, rate limits, authentication needs, or data freshness. For a weather API tool with zero annotation coverage, this leaves significant gaps in understanding operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that efficiently convey the core functionality and scope. Every word earns its place with no redundancy or fluff, making it easy to parse and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a weather forecasting tool. It lacks details on return format (e.g., what data fields are included), error cases (e.g., invalid location), and how it integrates with sibling tools, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (location and options with nested fields). The description adds no parameter-specific information beyond what's in the schema, such as format examples for location or default values beyond hours=24. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get hourly weather forecast for a location' with the specific scope 'hour by hour for the next 24 hours.' It distinguishes from siblings like 'getWeather' (likely current weather) and 'getWeatherForecast' (likely daily forecast) by specifying hourly granularity, though it doesn't explicitly name these alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (getWeather, getWeatherForecast, lookupCity). It mentions the default 24-hour scope but doesn't clarify if this is the only option or how it differs from other forecast tools, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getWeatherC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool 'Returns weather data including temperature, humidity, and conditions' which gives some output context, but lacks critical details like whether this is a read-only operation, if it requires authentication, rate limits, error handling, or what happens when location isn't provided (though schema covers this). For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that efficiently convey core functionality. First sentence states purpose, second describes return values. No wasted words, though it could be slightly more structured by separating usage guidance from behavior description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 100% schema coverage but no annotations and no output schema, the description provides basic purpose and return value information. However, for a weather API tool that likely has behavioral considerations (rate limits, authentication needs, error cases), the description should do more to compensate for the lack of structured metadata. It's minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters (location and options with units/language). The description adds no parameter-specific information beyond what's in the schema. Baseline 3 is appropriate when schema does all the parameter documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get current weather information for a location' with specific verb ('Get') and resource ('weather information'). It distinguishes from sibling tools by specifying 'current' weather (vs. 'hourly' or 'forecast'), but doesn't explicitly differentiate from 'lookupCity' which might be for location resolution rather than weather data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like getHourlyWeather or getWeatherForecast. The description implies it's for current weather, but doesn't state when to choose it over siblings or mention any prerequisites or exclusions. Usage context is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getWeatherForecastB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool 'Returns weather forecast for the next few days' which gives some output information, but doesn't describe error conditions, rate limits, authentication requirements, or what happens when location isn't provided (though the schema covers this). More behavioral context would be helpful for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that directly state the tool's purpose and what it returns. Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, 100% schema coverage, but no annotations and no output schema, the description provides basic purpose but lacks guidance on sibling tool differentiation and behavioral context. It's minimally adequate but has clear gaps in usage guidance and behavioral transparency that would help an agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description doesn't add any parameter semantics beyond what's in the schema - it mentions 'location' and 'forecast for the next few days' but these are already covered. The baseline of 3 is appropriate when the schema does all the parameter documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get weather forecast for a location' with the verb 'Get' and resource 'weather forecast'. It distinguishes from 'getHourlyWeather' by specifying 'for the next few days' rather than hourly, but doesn't explicitly differentiate from 'getWeather' or 'lookupCity'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus the sibling tools 'getHourlyWeather', 'getWeather', or 'lookupCity'. The description mentions 'Returns weather forecast for the next few days' which implies a multi-day forecast, but doesn't explicitly state this is for daily forecasts versus hourly or current weather.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookupCityB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool returns a city ID, which is useful, but doesn't cover other behavioral traits such as error handling, rate limits, authentication needs, or what happens with ambiguous inputs. For a lookup tool with zero annotation coverage, this leaves significant gaps in understanding its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by a practical note about the return value's use. Every sentence earns its place by adding clarity and utility without any waste or redundancy, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (a simple lookup), no annotations, no output schema, and high schema coverage, the description is adequate but incomplete. It covers the basic purpose and return usage but lacks details on behavioral aspects like errors or performance, which are important for a tool with no structured safety or output information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value beyond the schema by mentioning lookup methods (name, ID, coordinates) and the purpose of the returned ID. It doesn't provide additional syntax, format details, or examples beyond what's in the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Look up city information by name, ID, or coordinates.' It specifies the verb ('look up') and resource ('city information'), and mentions multiple lookup methods. However, it doesn't explicitly differentiate from sibling weather tools beyond noting the returned city ID can be used with them, which is helpful but not a full distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool returns a city ID for use with other weather tools, suggesting it's a prerequisite for those siblings. However, it doesn't provide explicit guidance on when to use this tool versus alternatives (e.g., direct weather queries) or any exclusions. The context is clear but lacks detailed alternatives or when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.