Server Quality Checklist

Tool Scores

• get-scan-detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves information, implying a read-only operation, but lacks details on permissions, rate limits, error handling, or what 'detailed information' includes (e.g., format, fields). This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'detailed information' entails (e.g., scan results, status, metadata), leaving the agent uncertain about the return values. For a tool with no structured output documentation, this gap is significant.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'scan_id' well-documented in the schema. The description adds no additional meaning beyond implying retrieval of a 'specific scan', which aligns with the schema but doesn't provide extra context like ID format or examples. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detailed information') and resource ('about a specific scan'), making the purpose immediately understandable. It distinguishes from the sibling 'list-all-scans' by focusing on a single scan rather than multiple scans, though it doesn't explicitly differentiate from 'run-nmap-scan' beyond the verb difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. The description implies usage for retrieving details of a known scan, but it doesn't mention prerequisites (e.g., needing a scan ID from 'list-all-scans') or contrast with siblings like 'run-nmap-scan' for creating scans.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run-nmap-scanC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Run an nmap scan') but doesn't describe what this entails—such as network impact, execution time, permission requirements, or output format. For a tool that likely performs network scanning, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—it directly states the tool's purpose without unnecessary details. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of running an nmap scan (a network operation with potential side effects), the description is incomplete. With no annotations, no output schema, and minimal behavioral context, it fails to provide enough information for safe and effective use. The agent lacks details on execution behavior, results, or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('target' and 'options') with clear descriptions. The description adds no additional meaning beyond what the schema provides, such as example usage or constraints, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Run') and resource ('nmap scan on specified targets'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'get-scan-details' or 'list-all-scans', which are clearly different operations, so it doesn't need explicit sibling differentiation for a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites like network access or permissions, nor does it explain when to choose this over siblings like 'get-scan-details' or 'list-all-scans'. This lack of context leaves the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-all-scansB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the action without details on permissions, rate limits, pagination, or response format. It doesn't clarify if this lists historical scans, real-time results, or includes metadata, leaving significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It's front-loaded and wastes no words, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and sibling tools, the description is incomplete. It doesn't address behavioral traits like safety, performance, or how results are returned, nor does it help differentiate from alternatives, leaving the agent with insufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, but this is appropriate given the schema's completeness, warranting a baseline score above minimum viable due to the lack of parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all available scan results'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get-scan-details' or 'run-nmap-scan' beyond the basic listing action, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-scan-details' for specific scan details or 'run-nmap-scan' to initiate new scans. It lacks context about prerequisites, timing, or exclusions, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.