Server Quality Checklist

Tool Scores

• domain_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It lists output categories (e.g., DNS records, reputation) but doesn't describe key behavioral traits like whether this is a read-only operation, if it requires authentication, rate limits, latency, or what happens with invalid inputs. The phrase 'supports identifying malicious or suspicious domains' hints at security analysis but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose ('Analyzes a given domain...') followed by specific insights. It uses two sentences efficiently, with the second sentence adding value by emphasizing security applications. There's minimal redundancy, though it could be slightly more structured (e.g., bullet points for output types).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (security analysis with multiple output categories), no annotations, no output schema, and 1 parameter with full schema coverage, the description is moderately complete. It outlines what the tool does but lacks details on behavioral traits, output structure, and usage context. For a security tool with rich potential outputs, more guidance on results and limitations would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 100% description coverage, providing a clear example value. The description adds no parameter-specific semantics beyond what the schema already documents (e.g., it doesn't explain format constraints or usage of the 'query' parameter). With high schema coverage, the baseline is 3, as the description doesn't compensate with additional param details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as analyzing domains to provide specific insights (DNS records, HTTP certificate details, reputation, security evaluations, tags, security classifications). It uses the verb 'analyzes' with the resource 'domain' and lists concrete output categories. However, it doesn't explicitly differentiate from sibling tools like 'hash_search' or 'url_search' beyond the domain focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'identifying malicious or suspicious domains effectively,' which implies a security context, but doesn't specify when to choose this over sibling tools like 'ip_address_search' or 'url_search' for similar security analysis. No explicit when/when-not statements or alternative tool references are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hash_searchB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions outputs like 'detects malware, suspicious activity, and clean results' but lacks details on rate limits, authentication needs, error handling, or whether this is a read-only operation. The description is informative but incomplete for safe agent invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences, front-loading the core purpose and listing key outputs. It avoids redundancy and wastes no words, though it could be slightly more polished for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides a good overview of the tool's function and outputs. However, it lacks details on behavioral traits (e.g., rate limits, auth) and doesn't fully compensate for the missing structured data, making it adequate but with clear gaps for a threat analysis tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for the single parameter 'query', with an example hash value. The description doesn't add any parameter-specific semantics beyond what the schema provides, such as hash format requirements or validation rules. Baseline score of 3 is appropriate given the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Provides detailed threat analysis for a given hash' with specific outputs like analysis date, reputation, categories, etc. It distinguishes from siblings by focusing on hash analysis rather than domains, IPs, or URLs, though it doesn't explicitly name the sibling tools for comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for threat analysis of hashes, but doesn't explicitly state when to use this tool versus the sibling tools (domain_search, ip_address_search, url_search). No guidance on prerequisites, alternatives, or exclusions is provided, leaving usage context somewhat ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• url_searchB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions outputs like 'reputation scores' and 'classifications from various sources,' but doesn't disclose behavioral traits such as rate limits, authentication requirements, data freshness, or error handling. The description is informative about what the tool returns but lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences, front-loading the core purpose and then elaborating on outputs and benefits. It avoids redundancy and stays focused, though minor trimming (e.g., 'effectively' at the end) could improve it further.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (threat analysis with multiple data sources), no annotations, and no output schema, the description is adequate but incomplete. It outlines what the tool does and its outputs but lacks details on response structure, limitations, or integration context, leaving gaps for an AI agent to infer behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'query' documented in the schema. The description doesn't add any parameter-specific details beyond what's in the schema (e.g., URL format expectations or validation rules), so it meets the baseline of 3 without compensating or detracting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Provides in-depth threat analysis for a given URL' with specific outputs like reputation scores, categories, and security evaluations. It distinguishes from siblings by focusing on URLs rather than domains, hashes, or IP addresses, though it doesn't explicitly name those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'threat analysis' and 'identify potential risks,' suggesting it's for security assessment scenarios. However, it doesn't explicitly state when to use this tool versus the sibling tools (domain_search, hash_search, ip_address_search) or provide any exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ip_address_searchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool does (analysis and summarization) and hints at security-related outputs, but does not cover critical aspects like rate limits, authentication needs, error handling, or whether it's a read-only or mutative operation. It adds some context but falls short of fully compensating for the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core function and listing key components. It uses two sentences efficiently to convey analysis scope and purpose, with no redundant information. However, it could be slightly more concise by avoiding minor repetition in listing details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (analysis of IP addresses with multiple data types) and lack of annotations or output schema, the description is moderately complete. It outlines what the tool does and its purpose, but does not detail return values, error cases, or behavioral constraints, leaving gaps that could hinder effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'query' parameter documented as an example IP address. The description does not add any meaning beyond this, as it does not explain parameter formats, constraints, or usage details. With high schema coverage, the baseline score of 3 is appropriate, as the description provides no extra parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Provides comprehensive analysis', 'Summarizes') and resources ('IP address'), listing detailed components like ASN, geographic location, WHOIS, and security evaluations. It distinguishes from siblings by focusing on IP addresses rather than domains, hashes, or URLs, making the scope explicit and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for threat detection and risk assessment, suggesting it's for analyzing IP addresses in security contexts. However, it lacks explicit guidance on when to use this tool versus alternatives like domain_search or url_search, and does not mention any exclusions or prerequisites, leaving usage context somewhat vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.