Server Quality Checklist

Tool Scores

• check_cveC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'Scan' and 'support status' but lacks details on permissions, rate limits, response format, or whether this is a read-only operation. For a security tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Scan for known security vulnerabilities and support status') with zero waste. Every word earns its place, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a security vulnerability tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits, response format, error handling, or how results are presented, which are crucial for an agent to use this tool effectively in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all parameters (product, version, vendor) with descriptions and examples. The description adds no additional meaning beyond what the schema provides, such as explaining how parameters interact or their impact on results, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Scan for known security vulnerabilities and support status' clearly states the tool's purpose with a specific verb ('Scan') and resource ('security vulnerabilities and support status'). It distinguishes from siblings like 'check_version' or 'compare_versions' by focusing on vulnerabilities rather than version checking, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'check_version' or 'get_all_details'. The description implies usage for vulnerability scanning but offers no explicit context, prerequisites, or exclusions, leaving the agent to infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_versionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'compare versions' implies a read-only analysis operation, it doesn't specify whether this requires authentication, has rate limits, returns structured data, or handles errors. The mention of 'detailed upgrade analysis' suggests richer output, but this isn't elaborated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just one sentence with zero wasted words. It's front-loaded with the core purpose and efficiently communicates the tool's function without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'detailed upgrade analysis' entails, how results are structured, or any behavioral constraints. Given the complexity implied by 'analysis' and lack of structured output documentation, this leaves significant gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already fully documents both parameters (product and version) with descriptions and examples. The description adds no additional parameter semantics beyond what's in the schema, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('compare versions' and 'get detailed upgrade analysis'), making it easy to understand what the tool does. However, it doesn't explicitly distinguish this from sibling tools like 'check_version' or 'get_all_details', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_version' or 'check_cve'. There's no mention of prerequisites, appropriate contexts, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_productsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'browse or search' implies a read-only operation, it doesn't specify whether this tool requires authentication, has rate limits, returns paginated results, or what the output format looks like (especially since there's no output schema). For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded with the core purpose ('Browse or search available software products'), making it easy for an agent to parse quickly. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for effective tool use. It doesn't explain what 'browse' entails (e.g., listing all products vs. paginated results), how search filtering works beyond the schema, or what the return values include (e.g., product names, IDs, versions). For a tool with no structured output documentation, the description should provide more context about expected behavior and results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'search' which aligns with the 'filter' parameter in the schema, but adds no additional semantic context beyond what the schema already provides (e.g., the schema's description covers 'Optional search term to filter products' with examples). With 100% schema description coverage and only one parameter, the baseline score of 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Browse or search available software products' clearly states the verb ('browse or search') and resource ('software products'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from its siblings like 'check_cve' or 'compare_versions', which appear to be more specific analysis tools rather than general browsing/searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, appropriate contexts, or comparisons to sibling tools like 'get_all_details' (which might return comprehensive product information). Without such guidance, the agent must infer usage from the tool name and schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_versionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Check' implies a read-only operation, it doesn't specify whether this requires authentication, has rate limits, returns structured data, or handles errors. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a simple lookup tool and gets straight to the point with no unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple lookup tool with good schema coverage but no annotations or output schema, the description is minimally adequate. It states what the tool does but doesn't provide enough context about behavior, alternatives, or expected outputs to be considered complete. The agent would need to make assumptions about how the tool actually works.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't add any parameter information beyond what's already in the schema (which has 100% coverage with clear descriptions and examples for both parameters). The baseline score of 3 is appropriate since the schema adequately documents the parameters, though the description could have added context about parameter relationships or validation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Check EOL status and support information') and resource ('software versions'), making it immediately understandable. However, it doesn't explicitly distinguish this from sibling tools like 'check_cve' or 'compare_versions' which might have overlapping domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_cve' (for vulnerabilities) or 'compare_versions' (for comparisons). There's no mention of prerequisites, typical use cases, or exclusions, leaving the agent to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_detailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves details but doesn't describe what 'comprehensive lifecycle details' includes (e.g., release dates, support status, vulnerabilities), whether it's a read-only operation, potential rate limits, or error handling. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, with every part contributing to understanding. This earns a 5 for optimal conciseness and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (retrieving lifecycle details for all versions) and lack of annotations and output schema, the description is incomplete. It doesn't explain what 'comprehensive lifecycle details' entails, how results are structured, or any behavioral traits. For a tool with no structured data beyond the input schema, this leaves too many unknowns for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single 'product' parameter with examples. The description adds no additional parameter semantics beyond implying the tool operates on a product, which the schema already covers. According to the rules, with high schema coverage, the baseline is 3 even without param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('comprehensive lifecycle details for all versions of a product'), making the purpose specific and understandable. It distinguishes from siblings like 'check_version' (single version) or 'list_products' (product names only), but doesn't explicitly name alternatives. This earns a 4 for clarity without full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when comprehensive lifecycle details across all versions are needed, as opposed to checking a single version or listing products. However, it doesn't explicitly state when to use this tool versus alternatives like 'check_cve' or 'compare_versions', nor does it provide exclusions or prerequisites. This results in a 3 for implied but not explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.