Server Quality Checklist

Tool Scores

• list_supported_tldsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool returns a 'complete list' but doesn't specify format, size, pagination, or performance characteristics. It doesn't address whether this is a cached list, real-time query, or has rate limits. For a read operation with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that directly state what the tool does and what it returns. Every word earns its place - there's no redundancy, fluff, or unnecessary elaboration. It's front-loaded with the core purpose and efficiently communicates essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with 1 parameter, no output schema, and no annotations, the description provides adequate but minimal information. It covers the core purpose and return value but lacks details about parameter usage, response format, and behavioral constraints. Given the low complexity, it's minimally viable but could benefit from more context about the 'limit' parameter and list characteristics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 1 parameter with 0% description coverage, so the description must compensate. While it doesn't explicitly mention the 'limit' parameter, it states the tool returns the 'complete list of TLDs', which implies the parameter might be optional or for pagination. The description adds meaningful context about what the tool returns, partially compensating for the schema gap. With 0 parameters documented in schema, baseline would be 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all supported TLDs') and resource ('TLDs that have WHOIS servers available'), distinguishing it from siblings like 'refresh_whois_servers' and 'whois_lookup' which perform different operations. It explicitly defines what the tool returns ('complete list of TLDs that can be queried'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'whois_lookup' or 'refresh_whois_servers'. It doesn't mention prerequisites, constraints, or typical use cases, leaving the agent to infer usage from context alone. While the purpose is clear, there's no explicit when-to-use or when-not-to-use information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whois_lookupA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the action (lookup via WHOIS protocol), scope (1,260+ TLDs), and data returned (registration details). However, it lacks details on error handling, rate limits, authentication needs, or network behavior, which are important for a network-based tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by supporting details. Every sentence adds value: protocol specifics, data returned, and TLD support. It's efficiently structured with no redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (network protocol tool with 2 parameters, no annotations, no output schema), the description is moderately complete. It covers the what and how but lacks details on output format, error cases, or performance considerations. For a tool interacting with external servers, more behavioral context would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It explains the 'domain' parameter implicitly by stating 'Look up domain information' and 'Queries... for domain registration details'. For 'include_raw', it's not mentioned, but with only 2 parameters and one clearly explained, this is adequate. The description adds meaningful context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Look up domain information using WHOIS protocol'), identifies the resource ('domain'), and distinguishes from siblings by focusing on domain lookup rather than listing TLDs or refreshing servers. It provides concrete details about the protocol (port 43) and scope (1,260+ TLDs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'domain registration details' and '1,260+ TLDs', suggesting it's for domain information retrieval. However, it doesn't explicitly state when to use this tool versus alternatives like list_supported_tlds or refresh_whois_servers, nor does it provide exclusion criteria or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_whois_serversA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the action (fetching from IANA, updating the dictionary) and suggests periodic use, but doesn't mention potential side effects, permissions needed, rate limits, or what 'updates' entail (e.g., overwriting, merging). It provides basic context but misses key operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with zero waste. The first sentence states the purpose, the second explains the effect, and the third provides usage guidance. Each sentence earns its place, and the information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with no annotations and no output schema), the description is somewhat complete but has gaps. It explains what the tool does and when to use it, but lacks details on behavioral traits (e.g., side effects, permissions) and output format. For a mutation tool with zero annotation coverage, more disclosure would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the input structure. The description appropriately doesn't add parameter details, as none are needed. It focuses on the tool's purpose and usage instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('refresh'), the target resource ('WHOIS server dictionary'), and the method ('fetching the latest TLD list from IANA'). It distinguishes this tool from sibling tools by focusing on dictionary maintenance rather than listing or performing lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('periodically to ensure the server list is up-to-date'), but it doesn't explicitly state when not to use it or mention alternatives. The guidance is helpful but lacks explicit exclusions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.