Server Quality Checklist

Tool Scores

• list_recent_criticalC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description hints at default behavior (7 days, high severity) but lacks details on pagination, output format, or other behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise, two sentences. Front-loaded with key purpose and default behavior. No redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, no parameter descriptions, no annotations. Description is too minimal for an agent to reliably understand the tool's full behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  0% schema description coverage. Description implies min_cvss and since_days roles via 'CVSS >= threshold' and 'default: 7-day', but doesn't explicitly explain either parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists recent CVEs with CVSS >= threshold and defaults to 7-day high-severity feed. Distinguishes from siblings (lookup_cve, find_for_package, severity_summary) by focusing on recency and severity filter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs siblings. Defaults are given but no conditions or exclusions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_for_packageA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It states the tool returns all CVEs but does not disclose pagination, ordering, coverage (e.g., only latest vs historical), or any restrictions. It provides minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short, front-loaded sentences with no unnecessary words. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters and no output schema, the description is adequate but lacks details on return format, ordering, or pagination. An agent may need more context to fully leverage the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 50% (only ecosystem has a description). The description mentions package_name and ecosystem as optional filter, adding context about optionality but not providing detailed semantics for package_name. It partly compensates for the missing schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists all CVEs for a given package with optional ecosystem filtering. The verb 'find' and resource 'CVEs affecting a package' are specific, and the sibling tools (e.g., lookup_cve for individual CVEs, list_recent_critical for recent critical ones) provide distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (for package-based CVE lookup) but does not explicitly state when not to use or mention alternatives. The sibling names give context, but the description itself lacks clear usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_cveA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It mentions 'cited NVD + GHSA URLs' but does not detail other behavioral traits like read-only nature, rate limits, or response size. For a simple lookup, this is minimally adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence (7 words) that front-loads the core purpose. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should explain return values. It only mentions 'Full record' and cited URLs, lacking specifics on what fields or structure the response contains. This is insufficient for a tool with no other docs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds no additional meaning beyond what is already in the schema's parameter description. It merely repeats the ID format example, failing to add value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Full record for one CVE by ID.' It specifies the resource (CVE) and the scope (one by ID), distinguishing it from sibling tools like 'find_for_package' which searches by package.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a specific CVE ID is known, but does not explicitly state when to avoid it or name alternatives. However, given the sibling tool names, context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• severity_summaryA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description only says 'counts' – does not explicitly state read-only or disclose other behavioral traits like data source or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise, two short phrases, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately describes purpose for a simple aggregation tool with no input or output schema; could specify which severity buckets are included.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so description does not need to add meaning beyond schema; baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses specific verb 'counts' and resource 'severity bucket across the snapshot', clearly distinguishing from siblings like lookup_cve or list_recent_critical.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicit usage hint via 'quick agent triage view', but no explicit when-to-use or when-not-to-use guidance compared to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.