Server Quality Checklist

Tool Scores

• diff_attack_pathsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavior disclosure. It accurately describes what the tool does (builds resource graphs, runs reachability searches, reports changes) and includes important limitations like 'Heuristic static analysis of declared IaC.' It also transparently explains the deep parameter's requirement for a CLOUD_PATHFINDER_KEY and the free tier behavior. It does not explicitly state it has no side effects, but as an analysis tool this is well implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-structured paragraph that leads with the core purpose, then explains the input shape, process, output verdict, and use case. While it is relatively long, every sentence adds valuable information for a complex tool. It is front-loaded with the key differentiator ('DIFFERENTIAL') and remains focused, earning a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the absence of an output schema, and no annotations, the description is impressively complete. It covers inputs (before/after IaC), the analytical process (graph building, reachability search), outputs (introduces/eliminates/aggravates, verdict types, deep-mode evidence), and usage context (PR gate). It also notes the heuristic nature and key requirements, leaving few gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema: it explains 'before' and 'after' as PR base and head trees with filename→content mapping, and clarifies the deep parameter's premium behavior (requires key, free version still yields verdict and counts). This enhances the schema's brief descriptions, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'DIFFERENTIAL attack-path analysis for a change to Infrastructure-as-Code' with a specific goal: reporting introduced, eliminated, and aggravated breach chains. It is specific about the verb, resource, and output verdict. However, it does not explicitly differentiate from the sibling tool 'audit_iac_attack_paths', so it misses the explicit sibling distinction that would warrant a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'the CI/CD gate', 'PR review', and explicitly contrasts it with 'a per-file linter or a single-state scan'. It states that this tool is needed for changes because it requires both graphs and semantic cross-state path matching. However, it does not mention when not to use it or compare directly with the sibling tool, so it lacks explicit exclusions/alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_iac_attack_pathsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses the heuristic nature: 'Heuristic static analysis of declared IaC,' explains it resolves cross-resource relationships, describes the output verdict and attack routes, and notes the API-key requirement for deep scans. It does not cover potential limitations like false positives or performance, but the core behavior is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-organized: it opens with the core purpose, explains the process, lists supported formats and attack primitives, provides usage guidance, and closes with the heuristic caveat. It is slightly long but every sentence adds useful information; the structure is front-loaded and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description fully explains the return value (verdict and routes) and the deep audit's additional evidence. It covers input flexibility, format auto-detection, cross-file resolution, API-key requirements, and known limitations. For a tool with 4 optional parameters and no output schema, the description is comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% description coverage for all four parameters. The description adds extra value by clarifying that 'files' can mix formats and cross-file references are resolved, and that 'source' is an optional alternative. It also explains the 'deep' parameter's premium behavior and key requirement, which goes beyond the schema's parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'Find real ATTACK PATHS in Infrastructure-as-Code' with concrete formats (Terraform, CloudFormation, Kubernetes) and differentiates it from 'a linter.' The verb is specific and the resource is clearly identified, making it distinct from the sibling tool 'diff_attack_paths' by focusing on full attack-path auditing rather than diffing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use it before applying IaC or in a PR to catch breach paths a per-resource linter misses.' It also explicitly excludes its primary alternative ('not a linter') and implies when to use it. However, it does not explicitly mention the sibling tool 'diff_attack_paths' or explain when to prefer this over a diff-based analysis, which would be a stronger guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.