Server Quality Checklist

Tool Scores

• get_forecastA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It explains a non-obvious behavior: dates are interpreted as local dates at the destination, not server dates. This provides useful transparency. It does not mention side effects, but as a read-only forecast tool, none are implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and informative, using examples and clarifications without unnecessary repetition. It flows logically from purpose to usage nuances to edge cases, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, key behaviors, and relationship to a sibling tool. Since there is no output schema, it does not need to detail return structure, and the input parameters are fully described in the schema. An agent has sufficient information to decide when and how to call the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptive text for all four parameters (startDate, endDate, location, days), covering their format and meaning. The tool description adds context about default behavior when no dates are given, but this does not enhance the understanding of individual parameters beyond the schema, so it remains at the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides daily weather forecasts with highs, lows, rain, and wind. It distinguishes itself from get_typical_weather by explicitly mentioning that tool for longer-range historical data, making the purpose unambiguous relative to at least one sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete guidance on when and how to use the tool: provide dates if known, otherwise the anchor is today at the destination. It explicitly directs users to get_typical_weather for trips further out. However, it does not explicitly contrast with get_hourly_forecast, leaving some ambiguity about when daily vs. hourly is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_typical_weatherA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It clearly states this is historical data, not a forecast, and explains that the year in the dates only fixes the calendar window while the summary covers complete past years. It also describes the return contents (average high/low, extremes, rain count). It lacks explicit statements about side effects or error conditions, but for a read-only historical query the description provides adequate behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first line states the core purpose and non-forecast nature, followed by usage context, return details, and a behavioral note. While it spans multiple paragraphs, each sentence contributes essential information without fluff. It could be tightened slightly but remains efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and four parameters, the description covers the essential return data (averages, extremes, rain count) and explains the critical usage context (forecast horizon and replacement by get_forecast). It addresses the year parameter's semantics and the historical nature. Minor gaps like error handling or explicit response format are not covered, but the description is sufficiently complete for an agent to invoke the tool correctly and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% description coverage for all parameters, setting a baseline of 3. The description adds semantic value beyond the schema, particularly for the year aspect: it clarifies that the year in the dates is used only to fix the calendar window and that the summary always covers complete past years. This nuanced interpretation is not in the schema and helps agents avoid misusing the year field. The description also implicitly explains the 'years' parameter by mentioning 'over recent years'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns observed historical weather data for a date window, explicitly contrasting with forecasts. It names the resource (typical weather) and the action (get), and distinguishes itself from sibling tools like get_forecast by highlighting the historical vs forecast nature. The purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use this tool: for trips beyond the 16-day forecast horizon, and explicitly states when not to use it — once dates fall within 16 days, get_forecast replaces it. It names the alternative tool directly and instructs not to keep quoting averages. This is a model example of usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_locationsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the burden of disclosing behavior. It clearly indicates a read-only search (finding places) with no mentions of destructive actions or side effects, but it doesn't explicitly state that it does not modify data or mention any authentication/rate-limit constraints. However, the nature of a search tool makes these omissions acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. Each sentence serves a purpose: the first defines the function, the second explains when to use it, and the third clarifies when not to use it. There is no redundant or extraneous information, making it easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides sufficient context for an agent to determine when and how to use this tool. It mentions the output includes coordinates and timezone, and it explains the relationship to sibling tools (that they accept place names directly). Without an output schema, the description does not need to detail return values, but it gives enough hints about the expected results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters (query and limit) with descriptions, providing 100% coverage. The tool description itself adds no additional semantic information beyond the schema; it only references the query parameter in the context of disambiguation. Since schema coverage is complete, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Find the places a name could refer to') and the resource (place names), clearly distinguishing it from the sibling tools like get_forecast. It also clarifies its role as a disambiguation step, which is unique among the provided tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly specifies when to use this tool: when a place name is ambiguous or when coordinates are needed for other tools. It also explicitly states when not to use it ('not as a routine preliminary step') and notes that other tools accept place names directly, leaving no ambiguity about its role.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_hourly_forecastA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns 24 rows, requires the date to be within the 16-day forecast horizon, and interprets the date as local at the location. This is strong transparency, though it doesn't detail error handling for out-of-range dates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with the core purpose stated first, followed by usage guidance and constraints. Every sentence adds value, and the length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is self-contained: it explains the return granularity, usage scenarios, alternatives, and date constraints. Even without an output schema, an agent knows what to expect and how to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context to the 'date' parameter by stating the 16-day horizon and local-date interpretation, which enhances the agent's understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides an hour-by-hour forecast for one day, specifically focusing on when rain arrives. It differentiates from the sibling get_forecast by emphasizing the single-day scope and the hourly granularity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool (when the day matters more than the week) and when not to, pointing to get_forecast for multi-day outlooks. Also warns about the 24-row return being wasteful for week-long questions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_best_daysA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It explains that scoring is over hours actually spent outside (8am–6pm), uses feels-like temperature that accounts for humidity and wind chill, and that the ranking is a stated preference, not a measurement. It also discloses that each day returns supporting numbers (feels-like range, rain chance, wind, humidity) and instructs the agent to report those alongside the pick.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but well-structured and front-loaded with the core purpose. Each paragraph adds distinct value: usage guidance, scoring logic, and output expectations. No filler or redundancy. Slightly verbose for the simplicity of the tool, but justified by the need to explain the scoring nuances.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description fully covers what the agent should expect and how to report results. It explains the ranking rationale, the parameters' influence, and the output format. There are no critical gaps that would leave an agent guessing how to invoke or interpret the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful context beyond the schema: it ties startHour/endHour to the outdoor hours window, explains that temperature parameters are feels-like, and clarifies that location can be omitted for home. This enhances parameter understanding without repeating schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Rank') and resource ('days ahead'), and immediately clarifies the purpose: picking a pleasant day for outdoor activities. It explicitly distinguishes this from 'what is the weather', which separates it from sibling forecast tools. The contrast with 'a day they have already chosen' further disambiguates it from get_forecast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use it when the user wants a day picked out of a range — a hike, a ride, a picnic, painting the fence — rather than the outlook for a day they have already chosen.' It also states what the tool answers ('when should I go') versus what it does not. This is direct and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.