Server Quality Checklist

Tool Scores

• clear_old_cacheA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description indicates a destructive action (clearing cache) and mentions the return value (status with count). However, it lacks details on side effects (e.g., impact on other users, reversibility) or safety considerations beyond 'expired' entries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear one-liner followed by structured args/returns. It could be even more concise but is well-organized and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, output schema exists), the description covers its core function and return value adequately. However, it does not elaborate on edge cases or behavior when no entries are expired.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the only parameter, 'max_age_minutes', beyond the schema's type and default, explaining it as 'Maximum age in minutes before considering expired'. This compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool action ('Clear expired weather cache entries') with a specific verb and resource, distinguishing it from sibling read tools like get_alerts, get_current_time, and get_forecast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for cache maintenance but does not explicitly state when to use this tool over alternatives (e.g., after forecast fetches) or provide any usage restrictions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_forecastA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses caching behavior (60 min expiry) and refreshing logic, adding value beyond missing annotations. However, it omits details on error handling, data format, or side effects, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences and bullet points, front-loading the core purpose. Every sentence contributes value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with an output schema, the description covers purpose, params with ranges, and caching. It is mostly complete but could mention behavior for invalid coordinates or when cache is not used.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds essential meaning by specifying valid ranges for latitude (-90 to 90) and longitude (-180 to 180). This compensates well for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a weather forecast for given geographic coordinates, using specific verbs and resource. It distinguishes well from sibling tools like 'clear_old_cache' and 'get_alerts' which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining a 5-day forecast with caching, but does not explicitly state when to use or when alternatives are better. No guidance on exclusions or prerequisites is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_alertsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses caching behavior (30 min expiry and refresh). Since no annotations are provided, this transparency helps the agent understand potential staleness. No destructive side effects are mentioned, which is appropriate for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (4 sentences) with clear structure: purpose, elaboration, caching behavior, and parameter details. Every sentence adds value without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter) and presence of output schema, the description covers all essential aspects: purpose, caching, and parameter format. It is complete enough for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description carries the full burden. It provides useful context: 'Two-letter US state code' with examples (CA, NY), which adds meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get active weather alerts for a US state', specifying the action and resource. It distinguishes from siblings like get_forecast or clear_old_cache by focusing on alerts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit when-to-use or when-not-to-use guidance compared to siblings. However, the purpose is clear enough that an agent would know to use it for alerts, not forecasts or cache operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_timeA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavior: it determines timezone via IP geolocation, handles missing IP by auto-detection or UTC fallback, and returns a formatted string with time, timezone, day, and ISO timestamp. No contradictions exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two short paragraphs), front-loaded with the purpose, and uses a clear docstring structure. Every sentence contributes value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one optional parameter, no nested objects, output schema present), the description is complete. It explains input behavior and the output format (formatted string with time, timezone, day, ISO timestamp) without needing to detail the output schema further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has only one parameter (ip_address) with no description, giving 0% schema coverage. The description adds full semantics: 'Optional IP address to determine timezone. If not provided or empty string, attempts to detect automatically or defaults to UTC.' This is essential context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the current time based on the user's IP address.' The verb 'get' and resource 'current time' are specific, and the method (IP-based geolocation) differentiates it from unrelated sibling tools like clear_old_cache and get_alerts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the use case (getting current time for a location) and the optional IP parameter. It does not explicitly state when not to use it or mention alternatives, but sibling tools are in different domains, so no confusion arises. The guidance is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.