Server Quality Checklist

Tool Scores

• ping_hostA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that it uses the system ping binary unprivileged, works on macOS/Linux, validates input strictly, clamps count to 1-10, and outlines return shapes. Minor omission: no mention of ICMP blocking or permission variability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently structured with Args/Returns sections. Every sentence adds value, though the first two sentences could be merged without loss.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with an output schema present, the description covers purpose, parameters, return values, platform constraints, and input validation. It lacks mention of rate limiting or ICMP-specific failure modes, but overall it is thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by explaining each parameter with examples, validation rules, and clamping behavior. For 'host', it specifies valid formats and rejection of shell metacharacters; for 'count', it states default and bounds.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Ping a host with ICMP echo requests and report packet loss and latency' and answers a specific question. This verb+resource combination distinguishes it from sibling tools like dns_lookup, http_health, and traceroute_host.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case ('Is this machine reachable at the network level?') but does not explicitly guide the agent on when not to use this tool or which sibling to choose instead. Alternatives are implied but not contrasted.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fleet_sweepA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full disclosure burden. It details concurrency (up to 10 at a time), list cap (50), default behavior, required conditions for port, and return format including error cases. Could add timeout specifics or rate limits, but overall transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose, then breaks down args and returns in a clear structure. A bit verbose on return details but each sentence adds value. Could be slightly tighter but still well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, zero schema coverage, and sibling context, the description covers inputs, behavior, constraints, and output format completely. It enables an agent to use the tool correctly without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description fully explains all three parameters: hosts (cap, URL handling), check (options and defaults), port (conditionality). Adds meaning beyond the schema's native types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Run one diagnostic check across a whole fleet of hosts, concurrently.' It clearly states the tool's capability and provides a signature question it answers, distinguishing it from sibling single-host tools like ping_host or tcp_port_check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the signature question and concurrency indication, but does not explicitly say when to use this over siblings. It lacks 'when-not' or alternative recommendations, e.g., if only one host, use ping_host instead. So only implied usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• traceroute_hostA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral transparency. It discloses that it uses the system traceroute binary with specific behavior: one probe per hop, 2-second wait, and returns an error (not an exception) if the binary is missing. This covers the main behavioral traits (cross-platform support, timeout handling, error mode). It doesn't mention side effects or network impact, but for a read-only network diagnostic tool, these are minor gaps. The description's detail compensates for the lack of annotations well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose sentence, a question to anchor the user, a brief technical implementation note, and then separate Args and Returns sections. It is front-loaded with the key action. Some redundancy exists (e.g., the return structure repeats the schema), but it earns its length by adding behavioral details not in annotations or schema. It could be slightly tighter, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has only 2 parameters, no annotations, and an output schema exists (which reduces the need to describe return values), the description is complete. It explains the parameters with behavioral details not in the schema (clamping, default, input types), describes the return format in full, and covers error handling. The description covers both normal and failure paths. The output schema exists, so not over-explaining return values is appropriate. For a low-complexity tool, this is comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully explain the parameters. It does: it describes 'host' as 'Hostname, IPv4, or IPv6 address' and 'max_hops' with clamping to 1..30 and a default of 15. This adds the clamping behavior and default value, which are not in the schema (schema only has default: 15 but no min/max). The description also explains the semantics of max_hops in the context of probing. It could further explain that the host parameter can be a domain or IP, but the current description is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Trace the network path to a host, hop by hop, with per-hop latency' using a specific verb ('Trace') and resource ('network path to a host'). It differentiates from siblings like ping_host (which tests reachability but not path) and dns_lookup (which only resolves names) by explicitly mentioning per-hop latency and the traceroute binary. The description also answers the question 'Where along the path does traffic slow down or die?', which clarifies its unique diagnostic role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (when you need to understand the network path hop-by-hop), but it does not explicitly mention when not to use it or name alternatives. However, the sibling tools are distinct (ping_host, dns_lookup, etc.), and the description's specific focus on path tracing makes the use case fairly clear. It lacks explicit exclusions like 'Do not use for simple reachability testing; use ping_host instead.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dns_lookupA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that A/AAAA lookups use the system resolver while CNAME/MX/TXT/NS use dnspython, and it reports resolution time. It does not cover rate limits or side effects, but for a read-only DNS lookup this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose statement, usage context, implementation detail, and separate Args/Returns sections. It is slightly verbose but every part serves a purpose. The key info is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (2 parameters, output schema present), the description is complete. It covers both success and failure response formats, distinguishes MX record structure, and includes error types (NXDOMAIN, timeout). The agent has enough context to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, leaving the description to fully explain parameters. It does so thoroughly, describing the 'name' parameter with an example, and 'record_type' with supported values and default. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool resolves DNS records and measures resolution time, with a clear framing question ('Does this name resolve, to what, and is DNS itself slow or broken?'). It distinguishes well from sibling tools like ping_host or http_health by focusing on DNS resolution and timing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description answers when to use the tool by defining its purpose and the specific questions it addresses. However, it does not explicitly state when not to use it or mention alternative sibling tools, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tcp_port_checkA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It covers key behaviors: returns states like 'open', 'closed', 'timeout', includes latency measurement, timeout clamping, and error conditions. It also explains what happens on unresolved hosts or invalid ports. This is comprehensive but could mention potential side effects (e.g., triggering firewall alerts) or idempotency. Still, given the zero-annotation context, it does a solid job.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise, using bullet points for parameters and returns. Every sentence adds value: the opening analogy and examples clarify purpose instantly. The format is well-structured with clear sections for purpose, args, and returns. No fluff or redundancy observed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, single-port diagnostic), the description is fully complete. It explains purpose, usage constraints, each parameter's meaning and constraints, return format for both success and failure, and differentiates from other tools implicitly. The output schema is provided, so return values need no further elaboration. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 3 parameters with 0% description coverage, so the description must add meaning. It does this effectively: 'host' gets clarification (hostname, IPv4, or IPv6), 'port' gets a single integer and range (1..65535), 'timeout' gets a default of 5 and clamping range. This adds significant semantic value beyond the raw schema, which only provides names and types. A minor deduction: the description could mention that timeout is optional (default handled), but it is essentially clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Check whether ONE TCP port on a host accepts connections.' This is a specific verb (check) with a distinct resource (single TCP port). It also explicitly distinguishes itself from a scanner, noting exactly one integer port per call, and provides real-world examples (e.g., 443 on a payment gateway). This differentiates it from sibling tools like ping_host or traceroute_host.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool ('Is the service listening?') and gives concrete examples. It also explicitly states what not to do: 'This is deliberately a single-port diagnostic, not a scanner: exactly one integer port per call; ranges and lists are rejected.' However, it does not explicitly name sibling tools as alternatives for broader scans (e.g., fleet_sweep) or other networking diagnostics (ping, dns_lookup). Thus, it provides clear context and exclusions but stops short of cross-referencing alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• http_healthA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains that the tool follows redirects, reports health (2xx/3xx), latency, redirect chain, and failure cases. It also mentions timeout clamping. This is transparent for a read-only GET tool, though it omits potential side effects like rate limiting or authentication handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a brief summary, then args, then returns. Every sentence is informative and earns its place. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema in the description, the description is complete. It covers input, output, and behavior, and the sibling tools are distinctly different. No missing critical information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters fully. It does: url is described as 'Full URL to check' with an example, timeout is described with clamping range and default. This adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'GET an http(s) URL, following redirects, and report response health' and answers specific questions about endpoint upness, speed, and redirects. It distinguishes from siblings like ping_host, dns_lookup, etc. by focusing on HTTP health checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies that only http:// and https:// URLs are accepted, which is a key constraint. The purpose is clear enough to imply when to use, but it lacks explicit differentiation from siblings (e.g., 'use this for HTTP endpoint health; for network connectivity use ping_host').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tls_cert_checkA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavior: it performs a verified TLS handshake, uses the system trust store, reports errors for expired/self-signed/wrong chain, and details both success and failure return structures. The 30-day expiration flag and error reason handling are explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a short purpose paragraph, then behavioral explanation, then parameter details, then return format. Every sentence adds value, and the information is front-loaded. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the absence of an output schema, the description fully covers all necessary aspects: parameters, success/failure responses, the 30-day threshold, and the verification process. It is complete and leaves no obvious gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description adds crucial meaning: 'host' uses SNI, 'port' is an integer 1-65535 with default 443. This goes beyond the schema's bare titles and types, fully compensating for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inspects TLS certificates and flags looming expiry, answering specific questions about expiration, issuer, and covered names. It distinguishes itself from sibling tools like ping_host, dns_lookup, and http_health by focusing on certificate inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description directly addresses the questions it answers ('Is this cert about to expire, who issued it, and which names does it cover?') and explains the verification process, implicitly guiding when to use it. However, it does not explicitly state when not to use it or name alternative tools for other scenarios, missing a clear exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.