Server Quality Checklist

Tool Scores

• detect-servicesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions using nmap's -sV probes and returns ScanResult, but does not disclose behavioral aspects like network noise, privilege requirements, or how errors are handled. The authorization warning adds some transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each earning its place. The purpose is stated first, followed by technical detail, an important warning, and return format. No redundancy or unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has five parameters and an output schema, the description provides sufficient context: method, returns, and an authorization warning. It does not cover edge cases like timeouts or empty results, but overall it is complete enough for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters. The description adds context about the nmap technique but does not provide additional semantics beyond the schema's parameter descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool detects services and versions on open ports using nmap's -sV probes. It specifies a concrete verb and resource, and distinguishes itself from sibling tools like scan-ports (which likely only lists open ports) and detect-os (which focuses on operating system detection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an authorization warning but provides no guidance on when to use this tool versus alternatives like quick-scan or scan-ports. It does not explain when service detection is appropriate or which scenarios favor this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• quick-scanA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It mentions 'aggressive timing for quick results' and returns a ScanResult, but does not disclose potential network impact, rate limits, or additional behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with 4 sentences plus a warning and return note. It front-loads the core purpose and avoids unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters and an output schema. The description covers the main functionality, authorization, and return type. However, it lacks explanation of how parameters like 'service_detection' affect results or performance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already documented. The description adds context like 'aggressive timing' but does not significantly enhance parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a 'fast scan of the most common ports' and 'top N most frequently used ports,' distinguishing it from sibling tools like 'scan-ports' or 'scan-vulnerabilities' which suggest broader or different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates it's 'good for getting a fast security overview' and includes an authorization warning, but does not explicitly state when not to use it or contrast it with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan-portsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions input validation to prevent command injection and returns a ScanResult, which is transparent. However, it does not disclose potential network disturbance, detection risk, or permission requirements beyond authorization.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with three main components: purpose, warning, return value. No superfluous sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters and an output schema, the description covers the core functionality and important usage warning. It is complete for the schema richness, though it could mention alternative tools for context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description does not add meaning beyond what the schema already provides for parameters. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (scan ports) and the resource (target host/network) using nmap. It distinguishes from siblings like detect-os or quick-scan by focusing on port scanning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an authorization warning but lacks explicit guidance on when to use this tool versus sibling tools like detect-services or quick-scan. The description implies usage for network reconnaissance but does not state exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover-hostsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It reveals the underlying tool (nmap -sn) and notes the authorization requirement, but does not disclose potential side effects (e.g., network noise, privilege requirements, or impact on targets).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (4 sentences plus a returns line) with no wasted words. It front-loads the core action, explains the method, provides use-case context, and includes a critical safety note. Every sentence adds distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the presence of an output schema, the description adequately covers what the tool does, how it works, and what it returns. It could mention prerequisites or potential limitations (e.g., network permissions), but overall it provides sufficient context for decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters have descriptions in the input schema (100% coverage), so the description adds no new meaning beyond schema. The description text itself does not elaborate on parameter semantics, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Discover live hosts on a network without performing a port scan' and specifies use of 'nmap ping scan (-sn)'. This clearly distinguishes it from sibling tools like scan-ports and detect-services, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description positions the tool as 'typically the first step in a network assessment' and emphasizes 'without performing a port scan', implying usage before port scanning. However, it does not explicitly state when not to use it or directly name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect-osA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Absent annotations, the description carries the full burden. It discloses use of nmap -O, required root/sudo, and authorization necessity. It does not detail network impact or side effects, but covers critical safety and behavioral traits well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three short paragraphs: clear purpose, usage details, and return type. Every sentence contributes meaningfully without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, 1 required, no enums, output schema exists), the description covers target type, timeout, privileges, and authorization. The output schema handles return value detail, so no gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description does not add additional meaning beyond what the schema provides, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Detect the operating system of a target host' using nmap TCP/IP fingerprinting, with a specific verb and resource. It distinguishes from sibling tools like detect-services or scan-ports by focusing solely on OS detection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes elevated privileges and authorization requirements, providing clear context for safe usage. However, it does not explicitly compare to sibling tools or state when not to use this tool, though it implies its specific purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan-vulnerabilitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full disclosure burden. It explains the use of safe script categories by default, inclusion of CVE identifiers, and the return type VulnScanResult. It also warns about triggering security alerts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and well-structured with a brief overview, important note, and return type section. Every sentence adds necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of 5 parameters and the existence of an output schema, the description adequately covers the tool's purpose, usage, and behavioral traits. It differentiates from siblings through specific vulnerability scanning focus.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by clarifying allowed script categories and default behaviors for ports, scripts, timing, and timeout, which is not fully captured in the parameter descriptions alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'scan' and the specific resource 'known vulnerabilities using Nmap Scripting Engine (NSE)'. It distinguishes from sibling tools like detect-os, detect-services, and scan-ports, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an important usage warning about authorization and potential security alerts. It implies usage for vulnerability scanning but does not explicitly compare to alternatives like quick-scan or scan-ports.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.