Server Quality Checklist

Tool Scores

• get_cve_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, openWorld, idempotent, and non-destructive behavior. The description adds value by specifying the return details (CVSS, EPSS, KEV, affected products), providing extra context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter, full schema coverage, an output schema present, and enough context in the description to explain what is returned, the description is complete for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the single parameter (cve_id) with 100% coverage and an example. The description adds minimal extra meaning beyond restating 'single CVE id', so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns full details for a single CVE id, listing specific data fields (CVSS, EPSS, KEV status, affected products). It is distinct from sibling tools like check_package_vulnerability, which focus on packages or batches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use when you need context on a specific CVE before recommending a patch or explaining risk.' It does not mention alternatives or exclusions, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_package_vulnerabilityA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate the tool is read-only, idempotent, open-world, and non-destructive. The description adds valuable context by explaining that outsideCoverage=true means no data and should be treated as unknown risk, not safe. It also clarifies the scope of coverage (infrastructure products and PyPI/npm packages), going beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that efficiently conveys purpose, usage, and important behavioral notes. It is front-loaded with the core action and includes contextual guidance without unnecessary verbosity. Slightly more structure (e.g., bullet points) could improve readability, but it is concise overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), return values are likely documented there. The description covers the tool's purpose, when to use it, coverage scope, and interpretation of a key response field (outsideCoverage). It implicitly differentiates from siblings by being a single-check tool, but explicit differentiation would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for both parameters. The description adds minimal extra meaning beyond examples in the schema; it mentions 'package or product slug' and 'exact version string', which are already present. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks for known CVEs and supply chain compromises, specifies the verb 'check', and identifies the resource as package or infrastructure product vulnerability. It explicitly mentions when to call it (before adding/updating dependencies), which distinguishes it from siblings like list_covered_products or get_cve_details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Call this before adding, updating, or recommending any npm, PyPI, or infrastructure dependency, including mid-conversation when a developer asks about installing or upgrading a package.' It also explains the meaning of outsideCoverage=true, helping the agent interpret results correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_batch_vulnerabilitiesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond the annotations: each item is billed as one API call, items outside coverage return 'outsideCoverage=true' and are not safe, and quota exceedance returns a 429 before any results are delivered without billing. Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, and the description does not contradict them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph with 4 clear sentences, each adding essential information. It begins with the core purpose, then details billing, usage context, and edge cases. While not split into sections, it is efficient and front-loaded. Slightly verbose for a very concise tool, but still well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but mentioned), 100% schema coverage, and rich annotations, the description covers all critical aspects: purpose, usage distinction, billing model, coverage edge case, and quota handling. No gaps remain for an agent to understand invocation and result interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all required and optional properties described with descriptions). The description reinforces the maximum of 100 items and the cost per item, and adds billing nuance and coverage behavior. This goes beyond the schema definitions, earning a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool checks vulnerabilities for up to 100 software packages or infrastructure products in a single request, using specific verbs ('check') and resource ('batch vulnerabilities'). It distinguishes itself from the sibling tool 'check_package_vulnerability' by targeting batch scenarios.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('instead of multiple check_package_vulnerability calls when you need to audit a lockfile, manifest, or dependency list') and provides guidance on handling items outside coverage ('treated as unknown risk, not safe') and quota limits ('429 returned before any results... no calls billed'). It does not explicitly mention when not to use it (e.g., single item), but the context implies it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_covered_productsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnly, openWorld, idempotent), the description adds behavioral context: it returns live vs. static data based on API key availability. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with main purpose, every sentence adds unique value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, rich annotations, and an output schema existing, the description fully covers behavior variations and links to sibling tools for edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description cannot add parameter meaning. Baseline 4 is appropriate as the description provides behavioral context (API key effect) that compensates for lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns Attested-covered products for CVE checks, distinguishing between live data with an API key and a static list without. It also differentiates from sibling tools like check_package_vulnerability by noting that PyPI and npm packages work with that tool even if absent from this list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when to use this tool (for CVE checks) and provides alternatives for checking packages not in the list via check_package_vulnerability. Also specifies behavior differences with and without an API key.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.