Server Quality Checklist

Tool Scores

• scan_fullA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false, so the description correctly implies a non-destructive scanning operation. The description adds context about output format (JSON + HTML), ranking criteria (EPSS, CISA KEV), and surfaced areas, but does not detail side effects or required permissions beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with numbered points and a comparison. It is front-loaded with the core purpose. However, it could be slightly more concise by removing redundant phrasing like 'these are a growing attack vector' which adds color but not essential instruction.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool (4 parameters, multiple surfaces), the description covers the scanning scope and output format adequately. It lacks parameter details and does not leverage the output schema (though not shown). The comparison with scan_project adds completeness. However, the missing parameter explanations reduce usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for missing parameter documentation. However, it only mentions project_path implicitly via the surfaces and output_dir in reports. It does not explain the purpose of output_dir, run_profile, or max_findings, leaving the agent unsure how to set them correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a comprehensive scan covering project dependencies, OS packages, and IDE extensions. It explicitly distinguishes from scan_project by noting that scan_full covers three surfaces that project-only scanners cannot reach.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use this tool (e.g., 'complete coverage', 'full audit') and when to use scan_project instead ('fast project-only checks'). Includes specific user request patterns and a warning about slower speed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_projectA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate openWorldHint (external access) and non-destructive. Description adds that it writes HTML reports to output_dir, which is a behavioral side effect. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with purpose and usage. Efficient, but could be slightly tighter. Length is justified by the added context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, usage, results (JSON+HTML), and alternatives. Output schema exists, so return values are documented elsewhere. Missing parameter descriptions are a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 4 parameters with 0% description coverage. Description only explains output_dir (reports written there). Other parameters like run_profile and max_findings are not described, so partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans project dependencies for CVEs and ranks by EPSS/KEV risk. It distinguishes from sibling scan_full by noting scope difference (npm/PyPI vs OS packages). Specific verb 'scan' and resource 'project dependencies' are present.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit best-for: wanting to fix CVEs first, supply-chain risk, pairing with validate_after_patch. Also recommends scan_full for broader coverage. Provides clear when-to-use and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_after_patchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, and openWorldHint. The description adds behavioral details: it compares two inputs and outputs resolved_count, remaining_count, new_count, and validation_passed, with condition for validation_passed. This adds value beyond annotations, but could include more on error behavior or input validation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two sentences, front-loaded with the core purpose, followed by usage guidance and return value explanation. No extraneous information; every sentence is necessary and effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's specific role, the description covers when to use, how to chain with siblings, and what outputs to expect. It is fairly complete but could mention input format expectations (e.g., must be output from scan_project) and handle cases of invalid input.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent (0% coverage). The description compensates by explaining that baseline_json and after_patch_json are scan results from before and after patch, and that they should be JSON strings. This provides essential meaning beyond the schema's bare field definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: confirming that dependency upgrades resolved CVEs. It specifies the action (validate), resource (dependency upgrades/CVEs), and distinguishes from siblings by noting uniqueness to TridentChain and referencing chainable usage with scan_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description gives clear when-to-use scenarios (after npm update, pip install -U, or patch application) and how to use it (chain with two scan_project calls or pass saved JSON results). It mentions uniqueness but lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.