Server Quality Checklist

Tool Scores

• analyze_codeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Scan' and 'Returns structured candidate records', implying a read operation but not explicitly confirming non-destructiveness, or any side effects, access requirements, or limitations. There is no mention of whether it modifies files or requires network access.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the primary action and outcome. It contains no redundant information and efficiently communicates the tool's core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with five parameters, no output schema, no annotations, and 15 sibling tools, the description is too thin. It fails to explain how to effectively use the tool, omits any notes on expected input formats or use cases, and does not help the agent decide between this and scan_file. The absence of usage guidance and behavioral context makes it incomplete for reliable tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description does not add any parameter-specific semantics beyond what is in the schema, which is acceptable given full coverage. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans source files for 0day vulnerabilities using specific pattern count, and mentions the output structure. It distinguishes from some siblings like generate_poc but does not explicitly differentiate from similar scan tools like scan_file, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any exclusions, prerequisites, or conditions for selecting this tool over siblings like scan_file or verify_finding. The agent is left to infer usage from the schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_fix_completenessC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It only states what the tool checks, not whether it is read-only, what it returns (e.g., boolean or report), whether it requires prior scanning, or how it handles missing candidates. This leaves significant uncertainty for an agent invoking the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action ('Check if a fix has been applied') and efficiently conveys the scope ('ALL vulnerable call sites') and the cross-referencing aspect. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description should provide more context about expected outputs, preconditions, and behavior with optional parameters. It does not mention what the tool returns, how 'known candidates' are obtained, or what happens if candidates are omitted. This is insufficient for an agent to understand the tool's full contract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all four parameters are documented. The description adds the notion of 'known candidates' which aligns with the 'candidates' parameter but does not provide additional semantics beyond the schema. Given full schema coverage, a baseline of 3 is appropriate; the description does not meaningfully enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check if a fix has been applied') and the resource ('at ALL vulnerable call sites in a file'), and adds the cross-referencing with candidates. It distinguishes itself from siblings like generate_fix or verify_finding by focusing on completeness across call sites, though it does not name alternatives explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description implies it is used after a fix is applied (to check completeness), but does not state prerequisites, exclusions, or why it would be preferred over verify_finding or scan_file. The context of use is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_fixB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, yet it only states what the tool does, not how it behaves. It does not mention whether it modifies files, returns a suggestion, requires network access, or has any side effects. The description also omits the existence of fallback JSON parameters (findingJson, candidateJson) and their interaction with structured inputs, leaving the agent guessing about important behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler. It front-loads the core action and lists the key inputs. It is not structured with headings or details, but for a tool with a simple one-line purpose, this is appropriately sized. Minor deduction for not being broken into more scannable parts, but efficiency is strong.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 parameters, nested objects, no output schema, and no annotations, yet the description offers only a high-level statement. It does not explain the output format, the relationship between required and optional parameters, the fallback mechanism, or any prerequisites (e.g., needing a verified finding). An agent cannot fully understand how to invoke or interpret the result from this description alone, making it incomplete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning by clarifying that the candidate contains line range and snippet ('candidate line range, actual snippet'), which the schema itself does not provide. However, it does not explain the purpose of findingJson/candidateJson fallbacks or why only filePath is required when the description implies all three sources are needed. No explicit conflict, but the description adds modest value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: generating a contextual fix based on specific inputs (candidate line range, actual snippet, verified finding). It uses a specific verb ('generate') with a clear resource ('fix') and names the key inputs, distinguishing it from siblings like generate_poc or generate_report. The purpose is unambiguous and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or any prerequisites (e.g., needing a verified finding from verify_finding first). It does not mention conditions that would select this tool over siblings, nor does it state any exclusions or dependencies. An agent must infer the workflow context from the tool's name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_pocB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not state whether the tool executes the PoC, writes files, or requires a sandbox/container environment; the mention of a Dockerfile only hints at containerization without clarifying side effects or execution behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-structured sentence that front-loads the core action and uses the rest to enumerate the template's concrete components. Every word adds meaning; there is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool, this is minimally viable: the purpose is clear, the parameter is fully documented in the schema, and the output components are enumerated. However, because there are no annotations and no output schema, the description should also clarify return format and whether any code is actually run, so significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with the cwe parameter already documented as 'CWE ID (e.g., CWE-78)', so the baseline is 3. The description adds only that the PoC is per-CWE, which aligns with the parameter but provides no additional format or value details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Generate') and a specific artifact ('per-CWE Proof-of-Concept template') while enumerating its contents: setup, trigger, expected output, cleanup, and Dockerfile. This clearly distinguishes it from sibling tools like generate_fix, which targets fixes rather than PoCs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives, and it never mentions sibling tools like generate_fix or scan_file. Usage is only weakly implied by the phrase 'per-CWE Proof-of-Concept template' and the surrounding tool list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_fileB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral context. It does disclose that the tool returns 'structured candidate records with grounding anchors (file hash + line count)', which is a meaningful behavioral trait. However, it does not explicitly state that the operation is read-only, requires no side effects, or mention any limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler or repetition. It front-loads the core action and resource, then states the output characteristics. Every part adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough top-level information for a single-file scanning tool: input, action, and output type. However, with no output schema and no annotation, it leaves questions about exact return structure, safety implications, and when to choose this over siblings. It is viable but not fully complete for an agent without further context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 as the schema already documents every parameter. The description adds minimal semantic value beyond the schema, only hinting at the output structure, not at parameter behavior or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Scan a single file for vulnerabilities' clearly identifies the input and action. It also describes the output as 'structured candidate records with grounding anchors', which adds clarity. However, it does not explicitly differentiate this tool from sibling tools like analyze_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives such as analyze_code or verify_finding. It states what the tool does but not the conditions that should lead an agent to select it. The phrase 'single file' faintly implies scope, but there is no explicit when-to-use or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_cvssB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention that the operation is read-only, nor does it describe error handling, input validation, or the exact output format. Although it appears to be a pure calculation, the description omits any information about side effects or failure modes, leaving the agent to make assumptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately communicates the action and scope. It is front-loaded with the main verb and resource, contains no filler, and every word contributes to its clarity. This is an exemplary model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with one parameter and no output schema. The description names the expected outputs (base score, severity, metric breakdown) and the schema defines the input format. While it does not specify the output structure in detail or address edge cases, the simplicity of the operation makes the description reasonably complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage of the single parameter with a clear description and an example. The description simply restates that the tool takes a vector string, adding no additional meaning beyond what the schema already offers. It meets the baseline expected when schema coverage is high but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (calculate), the resource (CVSS v3.1 base score, severity, and metric breakdown), and the input (vector string). This is specific and distinguishes it from the sibling tools, which focus on fix generation, scanning, and other unrelated tasks. No ambiguity remains about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. Even though it is the only CVSS-related tool among the siblings, it does not explicitly mention use cases, prerequisites, or conditions under which it should be selected. This lack of usage context is a notable gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_reportB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that a report is generated but does not specify side effects (e.g., whether it writes files, returns output, or requires authentication), output format, or any operational constraints. For a tool with no annotation coverage, this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The main purpose is front-loaded, and the mode breakdown is efficiently presented. Every element contributes to the agent's understanding without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters and no output schema, the description is incomplete. It does not explain how to choose between findings/findingsJson or candidates/candidatesJson, nor what each mode's report entails beyond a label. An agent would likely need to infer or experiment to correctly call the tool, especially since only mode is required and the description implies findings are necessary but doesn't enforce it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover 63% of parameters, and the description adds value by explaining the mode parameter's meanings (A/B/C), which the schema only labels as 'Report mode'. However, it does not clarify other parameters like ecosystem, projectName, projectVersion, or the distinction between findings and findingsJson. The added value is partial—mode gets enriched but others remain opaque.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a security report from verified findings, with a specific resource (report) and verb (generate). It distinguishes three modes (A, B, C) that map to different report types, making the purpose unambiguous and differentiating it from sibling tools like generate_fix or scan_file, which have different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used after findings are verified and provides mode options, giving context for when to use it. However, it does not explicitly state when not to use it, nor does it mention alternatives or prerequisites. The guidance is implicit rather than direct, so an agent must infer the appropriate usage context from the 'from verified findings' phrase.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_patternsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only says 'List' and mentions filtering options, which repeats the schema. It does not disclose whether the operation is read-only, whether results are paginated, how filters combine (AND/OR), or the response format. This is a minimal disclosure insufficient for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action ('List vulnerability detection patterns') and then lists the filtering dimensions. There is no filler or redundancy. It is appropriately concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic purpose and filter options, but lacks critical contextual details such as output format, pagination, or filter combination behavior. Since there is no output schema, the agent is left guessing about return values. For a simple listing tool, it is adequate but not complete; more information about how filters interact would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter clearly described (e.g., 'Filter by CWE ID'). The description merely restates the existence of filters without adding new meaning. No additional semantics like value formats, combination rules, or wildcard behavior are added. Baseline 3 is appropriate given the schema already documents parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List vulnerability detection patterns') and names the resource. It clearly indicates filtering capabilities, distinguishing it from sibling tools that perform actions like generating fixes or scanning files. The verb 'List' and object 'vulnerability detection patterns' are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving patterns with optional filters, which is clear context. However, it provides no explicit when-not-to-use guidance or alternatives. The sibling 'list_fix_templates' could confuse an agent, and the description does not distinguish between listing patterns vs. fix templates. Usage scenario is implied but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_pipelineA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses two key side effects: writing persistent artifacts to a run directory and returning a compact executive summary. However, it does not mention error behavior, reversibility, or resource cost, and 'resilient' is left undefined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with the action sequence front-loaded and no fluff. The core purpose and major outputs are stated efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, no output schema, and no annotations, the description gives a high-level overview but omits practical details: what the executive summary contains, how parameters interact, or what happens when pipeline stages fail. For a complex orchestration tool, this is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 75% of parameters, and the description adds no extra meaning to any of them. It doesn't explain how reportMode, minPriority, or projectRoot influence the pipeline, leaving the agent to rely solely on the schema's minimal descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb ('Run') and resource ('full resilient 0day audit pipeline'), and enumerates the exact stages (Scan → Verify → Chain → Report → Fix). It clearly distinguishes this from sibling tools like scan_file, verify_finding, and chain_findings, which are individual stages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the comprehensive orchestrator, but it never explicitly states when to use it versus alternatives, nor does it mention exclusions (e.g., 'if you only need a single stage, use scan_file'). The context is clear from the sibling list but not made explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chain_findingsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It conveys useful behavioral traits: input must be pre-verified, output is multi-step and publication-grade. But it does not disclose side effects (e.g., whether saveToWorkspace persists files), return format, or what happens when findings cannot be chained — gaps that matter for a synthesis tool with no annotation safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence that names the action, the input class, and the output, reinforced by two illustrative chain examples. Every element earns its place with no filler or repetition of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The core purpose is well covered, but for a 6-parameter tool with no annotations and no output schema, the description omits practical details an agent needs: what format the chain artifact takes, whether it persists to the workspace, and behavior on unchainable findings. It is adequate but not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all six parameters are already documented and the baseline of 3 applies. The description adds no parameter-level meaning, notably leaving the object vs. JSON-string fallback relationship (findings/findingsJson, candidates/candidatesJson) unexplained, though that is not the description's obligation at this coverage level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (synthesize verified findings) and a specific resource (multi-step, publication-grade exploit chains), and grounds it with concrete examples (Traversal + Write -> Persistent RCE; Header Injection -> Agent Escape). This clearly distinguishes it from siblings like verify_finding (single-finding verification), generate_poc, and generate_report — nothing else in the sibling set composes findings into chains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The qualifier 'verified findings' implies use after verify_finding, and the examples clarify the intended composition scenario. However, the description gives no explicit when-to-use/when-not-to-use guidance or named alternatives — an agent must infer that this is for multi-finding composition rather than for generating a single POC or report.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_platformA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It lists the types of information detected, which gives some transparency about what the tool inspects, but it does not disclose the output format, whether it performs any system scans, or any potential side effects. The absence of an output schema makes this gap more significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the verb and enumerates a comprehensive list of detected attributes. It is concise but packed with useful detail, earning a high score for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although the description enumerates the detected aspects, it does not describe the structure or type of the returned data, which is critical since there is no output schema. An agent would benefit from knowing whether the tool returns a JSON object, a string, or individual values. This lack of output specification leaves the tool incomplete for reliable usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the schema trivially covers everything. The description adds nothing beyond the schema because there is nothing to add. Baseline of 4 applies for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Detect') and enumerates the exact aspects of the host environment it covers (OS, architecture, shell, homedir, path separators, permission commands, and OS-specific vulnerability nuances). This makes the scope unambiguous and distinguishes it from sibling tools that focus on code analysis, fixing, and reporting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for gathering platform information, but it does not explicitly state when to use it relative to other tools, nor does it mention any exclusions or prerequisites. An agent would need to infer that this is a preliminary detection step, but no explicit guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• profile_targetA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. Words like 'detect' and 'profile' strongly imply a read-only operation, and the mention of hashes and line counts suggests file reading without modification. Yet it does not explicitly state that the tool does not modify the project, nor does it address permission or performance considerations. The implicit signal is present but not fully explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action ('Profile a target project') and lists concrete findings. It includes the phase context without wasted words. While it could be slightly more structured (e.g., bullet-like), it is efficient and readable, earning a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description must convey what the agent will receive. It lists the detected elements but not the format or structure of the result. It also omits prerequisites (e.g., project root must exist) and any integration with downstream audit steps beyond being 'Phase 0.' For a tool of this complexity, this is a moderate gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter description coverage (projectRoot, include, exclude all have clear descriptions). The tool description adds no additional meaning about these parameters, leaving semantics entirely to the schema. Since the schema already explains them well, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool profiles a project and enumerates specific outputs (ecosystem, languages, frameworks, entry points, manifest files, file manifest with hashes and line counts). It also positions itself as 'Phase 0 of the audit pipeline,' distinguishing it from siblings like detect_platform or audit_pipeline. The verb-resource pairing is explicit and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Phase 0 of the audit pipeline' gives clear placement in a workflow, implying it should be used first before other audit steps. However, it does not explicitly name alternatives or define when not to use it (e.g., versus detect_platform). That omission keeps it from a 5, but the context is strong enough for a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_reachabilityA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden. It discloses the returned data (reachability status, path, etc.) and implies a read-only analysis by using 'trace', but does not state whether it modifies anything, requires special permissions, or has side effects. This is partial transparency but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, dense sentence that front-loads the action and lists the key outputs without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's purpose and return values, but does not clarify the optional parameters (candidate vs. candidateJson, projectRoot), handling of absent candidate, or potential prerequisites. Given no output schema, an agent might not know how to construct the optional candidate object or when to rely on filePath alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all parameters, so the schema already provides meaning. The tool description adds no further parameter semantics, such as explaining the relationship between candidate and candidateJson or when to use projectRoot. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('trace data flow'), a specific resource ('scan candidate'), and enumerates the output components (reachability status, data-flow path, auth gates, sanitizers, confidence). It clearly differentiates from siblings like scan_file or analyze_code by focusing on reachability analysis for a candidate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context by specifying it applies to a scan candidate, but does not explicitly mention when to avoid it or name alternative tools. It provides context without exclusions, matching the 'clear context, no exclusions' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_findingA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure. It does list the steps performed (reachability check, confounder elimination, CVSS calibration, PoC template generation) and states a requirement for CONFIRMED status. However, it does not disclose side effects (e.g., does it modify files?), what happens on failure, or any permissions required. It also doesn't mention whether the 'per-CWE PoC template' is written to disk or returned in the response. This is moderate disclosure, leaving important aspects implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence front-loads the verb and object, then lists the actions in a compact list. The second sentence adds a critical prerequisite. Every word earns its place, and it is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, one a nested object) and no output schema, the description should explain what the tool returns or produces. It mentions generating a PoC template but does not specify whether that is returned as a string, written to a file, or included in the response. It also doesn't explain the possible outcome statuses beyond mentioning 'CONFIRMED' without elaboration. The description covers the process but not the result, leaving an agent uncertain about the tool's full behavior. It is adequate but incomplete for a tool this complex.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all parameters, so the schema already documents each parameter (filePath, candidate, projectRoot, candidateJson). The description adds general context about the verification workflow (e.g., 'Requires high/medium confidence and cleared confounders') but does not provide parameter-specific semantics beyond what the schema offers. Satisfaction of prerequisites is mentioned, but it doesn't map directly to any parameter's format or usage. Since the schema covers the parameters well, the description's additional value is limited, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Verify a scan candidate') and lists distinct steps: reachability, 7-class confounder elimination, CVSS calibration, and per-CWE PoC template generation. This distinguishes it from sibling tools like check_reachability, calculate_cvss, and generate_poc, which each handle only one of these aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this tool is for verifying a scan candidate through a multi-step process. It includes a prerequisite for achieving CONFIRMED status (high/medium confidence and cleared confounders), which guides usage. However, it does not explicitly state when to use this tool instead of calling the individual sibling tools (e.g., 'use this instead of separate calls to check_reachability, calculate_cvss, etc.'), so it lacks explicit exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_fix_templatesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It indicates a read-only listing operation ('List') and mentions the fields returned (fix class, description), but does not disclose potential side effects (likely none), ordering, pagination, or error handling. For a simple list, this is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence that front-loads the action and the output fields. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description conveys the core functionality and explicitly mentions the fields returned (fix class, description). It is sufficient for an agent to call it correctly, though it could optionally mention ordering or any filtering mechanism.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter-specific information because there are none to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('all available per-CWE fix templates'), and specifies the information included ('with their fix class and description'). This clearly distinguishes it from sibling tools like generate_fix which perform a creation action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is for browsing available fix templates before generating or applying one, but it does not explicitly state when to use this tool versus alternatives such as generate_fix. There is no explicit when-not-to-use or alternative mention.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.