Server Quality Checklist

Tool Scores

• get_weather_forecastA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It states it returns 'detailed hourly weather forecast' but doesn't disclose data sources, units, location matching behavior, potential errors, or any other operational details. It adds little beyond the core purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and object. It is appropriately concise with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple weather tool with schema-covered parameters and no output schema, the description provides adequate context to understand the tool's purpose. However, it lacks any mention of return value shape, units, or edge cases, making it minimally viable but not richly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both 'location' and 'hours' having descriptions. The description's mention of 'hourly' loosely maps to the hours parameter but doesn't add format or constraints beyond what the schema already provides. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states a specific verb ('Get'), a resource ('weather forecast'), and scope ('hourly', 'UK location'). It distinctly separates this tool from siblings like get_travel_info and get_outfit_recommendation by domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the name and description; it's obviously for weather forecasts. However, there is no explicit guidance on when to use it vs alternatives, nor any exclusions or prerequisites beyond the required location parameter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_travel_infoA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the transparency burden. The verb 'Get' signals a non-destructive read operation, which is useful, but the description does not disclose any limitations, default behavior, or side effects. It is neither misleading nor exceptionally informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no redundant terms or filler. It communicates the core purpose efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no output schema, and no annotations. The description covers the main function but omits the mode parameter's behavior and return format details. For a simple tool, it is adequate but leaves clear gaps given missing annotation support.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67% (origin and destination have descriptions; mode does not). The description adds no semantic value beyond the schema—'between two locations' merely restates origin/destination, and the 'mode' parameter is left unexplained in both description and schema. The default of walking is only in the schema, not the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a clear resource 'travel time and distance between two locations.' It fully distinguishes the tool from siblings like weather or outfit recommendations, leaving no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for travel-related queries but provides no explicit when-to-use guidance or exclusions. Given that sibling tools are unrelated, there is no direct alternative to contrast, so the guidance is merely implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_user_preferenceA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of disclosing behavior. It only says 'Store' and gives examples; it does not mention whether existing keys are overwritten, what the tool returns, or any prerequisites or side effects. This is a significant gap for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and includes a 'Use this when' clause. Every word earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with a well-described schema, the description provides enough context to invoke it correctly. It gives clear usage triggers and examples. However, it omits behavioral details like overwrite semantics and return value, which would be relevant for a complete picture, but given the simplicity, the gap is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add significant meaning beyond the schema; its examples ('home' as key, 'Bristol' as value) merely mirror what the schema already provides, so no extra credit is earned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Store a user preference like home/work addresses or other settings.' It uses a specific verb ('store') and resource ('user preference'), and gives concrete examples that distinguish it from the sibling get_user_preferences tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this when the user wants to save a location for future use' and provides examples of when to apply it. However, it does not explicitly mention when not to use it or name the alternative getter tool as an exclusion, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_outfit_recommendationA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It explains that the tool analyzes four factors to generate a recommendation, which provides some transparency about its internal logic. However, it does not disclose potential side effects, dependencies on external services, or what happens if data is unavailable. For a simple get-type tool, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence immediately states the tool's purpose, and the second clarifies the analysis logic. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is present, and the description does not describe the return format (e.g., text vs. list of items) or any edge cases. It covers the core functionality and inputs but leaves the exact response shape ambiguous. Given the moderate complexity, this is acceptable but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so parameters are well documented. The description's mention of 'location, destination, travel time' maps to origin/destination/hours_until_departure but adds no extra semantic detail beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get clothing recommendations' with a specific verb and resource, and explains the inputs it uses ('current location, destination, travel time, and weather forecast'). This distinguishes it from sibling tools like get_weather_forecast and get_travel_info which focus on single data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need clothing advice informed by weather and travel plans. It does not explicitly name alternatives or exclusion scenarios, but the distinct purpose ('suggest what to wear') is clear enough for an agent to choose this over the weather-only or travel-only siblings. However, it lacks explicit 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_preferencesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It clearly indicates a read-only operation via 'Retrieve' and clarifies the optional key behavior, but it does not mention return format, error handling, or what happens if a key is not found. These gaps are minor for a simple getter but leave some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and every sentence earns its place. It is appropriately sized for the tool's simplicity with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one optional parameter and no output schema, the description adequately explains the core functionality and even gives a practical use case ('see what locations the user has saved'). It does not detail the return structure, but for a simple preferences getter this is not a significant omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains the 'key' parameter well. The description merely restates what the schema says ('get a specific preference by key' and 'If omitted, returns all preferences') without adding extra meaning like examples or default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb 'Retrieve' with a clear resource 'user preferences', and explains both modes: all preferences or a specific key. This clearly distinguishes it from sibling tools like get_weather_forecast or set_user_preference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a concrete use case: 'Use this to see what locations the user has saved.' It does not explicitly exclude alternatives or mention when to use set_user_preference instead, but the context is clear enough for a simple getter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.