Server Quality Checklist

Tool Scores

• search_newsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions searching 'titles and descriptions' and 'all Icelandic news sources', but doesn't cover important aspects like rate limits, authentication requirements, pagination behavior, error conditions, or what happens when no results are found. For a search tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - two clear sentences with zero wasted words. The first sentence establishes the core functionality, and the second adds important scope information about searching titles and descriptions. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there's an output schema (which means return values are documented elsewhere), the description covers the basic purpose adequately. However, for a search tool with 4 parameters and no annotations, it should ideally provide more context about behavioral aspects like result ordering, freshness, or limitations. The description is minimally complete but lacks depth.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'keyword or phrase' which aligns with the query parameter, but doesn't provide additional context about parameter interactions or search semantics. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search across all Icelandic news sources for articles matching a keyword or phrase.' It specifies the verb (search), resource (Icelandic news articles), and scope (across all sources). However, it doesn't explicitly differentiate from sibling tools like 'get_news' or 'check_feeds', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_news' or 'check_feeds'. It mentions searching 'titles and descriptions' but doesn't clarify if this is the primary search method or when other tools might be more appropriate. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_feedsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool checks 'health and availability' but doesn't specify what that entails (e.g., HTTP status codes, response times, error details), whether it performs network requests, potential side effects like caching, or rate limits. The description adds minimal context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences: the first states the purpose, and the second provides usage context. It's front-loaded with the core functionality. However, the second sentence ('Useful for debugging and monitoring.') could be more integrated or specific to enhance clarity without adding waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which handles return values), 2 parameters with full schema coverage, and no annotations, the description is minimally adequate. It covers the purpose and basic usage but lacks details on behavior, error handling, or integration with siblings. For a monitoring tool, more context on what 'health' entails would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters ('sources' and 'timeout'). The description doesn't add any parameter-specific semantics beyond what the schema provides (e.g., it doesn't explain what 'health' means in relation to 'sources' or how 'timeout' affects the check). Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check the health and availability of RSS feeds' with specific verbs ('check', 'health', 'availability') and resources ('RSS feeds'). It distinguishes from siblings like 'get_news' or 'search_news' by focusing on monitoring rather than content retrieval. However, it doesn't explicitly differentiate from 'list_feeds', which might be a related listing operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context: 'Useful for debugging and monitoring.' This suggests when to use it (for health checks) but doesn't explicitly state when not to use it or name alternatives. For example, it doesn't clarify if this should be used instead of 'list_feeds' for availability checks or how it relates to 'get_news' for content access.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_feedsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists feeds and returns structured metadata, which implies a read-only operation, but doesn't cover aspects like rate limits, authentication needs, error handling, or pagination. This is adequate for a simple list tool but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste: the first states the purpose and scope, and the second specifies the return type. It's front-loaded and appropriately sized for a simple tool, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers the core functionality and return type, though it could benefit from more behavioral context or usage guidance to reach a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, detailing the 'source' parameter with enum values and a default. The description adds no parameter-specific information beyond what the schema provides, so it meets the baseline of 3 for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all available news feeds') and resource ('from Icelandic news sources'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_news' or 'search_news' which might have overlapping functionality, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_news' or 'search_news'. It mentions the return type ('structured feed metadata') but offers no context about use cases, prerequisites, or exclusions, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cache_statsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's purpose and utility but lacks details on behavioral traits like permissions needed, rate limits, error conditions, or what the output looks like (though an output schema exists). The description doesn't contradict annotations (none provided), but it's minimal beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second adds context without redundancy. Every sentence earns its place by providing essential information efficiently, with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects), high schema coverage (100%), and the presence of an output schema, the description is reasonably complete. It covers the purpose and utility, though it could benefit from more behavioral context (e.g., performance implications). The output schema reduces the need to explain return values, making this adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter semantics beyond what the input schema provides. The schema has 100% description coverage for its single parameter ('clearCache'), clearly explaining its effect. With high schema coverage, the baseline is 3, as the description doesn't need to compensate but also doesn't add value regarding parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get cache statistics') and resources ('cache statistics including hit/miss rates and entry counts'). It distinguishes itself from sibling tools (which are about feeds/news) by focusing on cache monitoring. However, it doesn't explicitly differentiate from potential cache-related siblings if they existed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance with 'Useful for monitoring performance,' suggesting when this tool might be appropriate. However, it doesn't explicitly state when to use it versus alternatives, nor does it mention any prerequisites or exclusions. The sibling tools are unrelated (feeds/news), so no comparative guidance is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_newsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool fetches news, supports date filtering, and returns structured data, covering basic operation and output. However, it lacks details on rate limits, authentication needs, error handling, or whether it's a read-only operation (implied by 'fetch' but not explicit). The description adds value but misses key behavioral traits for a tool with 6 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded: the first states the core purpose and sources, the second adds key features (date filtering, return format). Every sentence earns its place by providing essential information. It could be slightly more structured by explicitly separating functionality from output details, but it's efficient with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, no annotations, but with output schema), the description is reasonably complete. It covers what the tool does, sources, key features, and return data. With an output schema present, it doesn't need to explain return values in detail. However, for a news-fetching tool with multiple parameters and siblings, it could benefit from more explicit behavioral context or usage examples.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 6 parameters with descriptions, defaults, and constraints. The description adds minimal value beyond the schema, mentioning 'supports date filtering' (hinting at since/until parameters) and listing some sources, but doesn't explain parameter interactions or provide additional semantic context. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('fetch', 'returns') and resources ('Icelandic news', 'NewsItem data'). It distinguishes from siblings by specifying it fetches 'latest' news from specific sources with date filtering, unlike search_news (likely for searching), list_feeds (listing feeds), check_feeds (checking status), and cache_stats (monitoring cache).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to get the latest news with date filtering from Icelandic sources. It implicitly distinguishes from search_news by not mentioning search functionality, but lacks explicit guidance on when to choose this over search_news or other siblings. The input schema references list_feeds for feed options, adding some usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.