Server Quality Checklist

Tool Scores

• analyse_spfA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It discloses the analysis scope (full include tree, DNS lookup count, authorized senders) and specific misconfigurations it flags (multiple SPF records, +all). This is substantive enough for an analysis tool, though it does not mention network behavior or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences. The first sentence front-loads the primary action and enumerates specific checks. The second sentence provides direct use cases. Every clause adds value, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter analysis tool with no output schema, the description explains the main behaviors and common triggers for use. It does not describe the exact output format, but the listed deliverables (lookup count, authorized services, flags) give a sufficient mental model for an agent to decide invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% coverage on the single 'domain' parameter, including the example and instruction to exclude scheme/path. The description restates that it analyzes a domain's SPF record but does not add extra parameter-level detail beyond what the schema provides, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Analyse a domain's SPF record' and then lists concrete capabilities (resolve include tree, count DNS lookups, identify authorized senders, flag misconfigurations). This clearly distinguishes it from sibling tools like scan_domain and check_tls by focusing exclusively on SPF analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases: 'Use this for SPF PermError, "too many DNS lookups", or to find out which providers can send mail as a domain.' This provides clear context for when to choose this tool, though it does not mention when not to use it or name any alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_domainA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable context by stating the tool returns 'live results from public DNS records and TLS handshakes,' indicating a read-only, real-time scan. It also lists the specific security checks performed, giving the agent a clear model of behavior. However, it doesn't mention any potential side effects, rate limits, or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: the first states the core purpose and scope, the second provides usage triggers, and the third explains the method and return type. There is no redundant content, and each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple single-parameter schema and no output schema, the description provides sufficient context for an agent to select and invoke the tool correctly. It covers the tool's purpose, when to use it, what it checks, how it works, and what it returns. This is complete for the tool's complexity and complements the sibling tools effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the 'domain' parameter with 100% coverage, including an example and constraint ('Do not include a scheme or path'). The description adds no new semantic detail beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('a domain's full security posture'), and enumerates concrete components (SPF, DKIM, DMARC, MTA-STS, TLS, HTTP headers). It distinguishes itself from sibling tools like check_tls and analyse_spf by offering a comprehensive security review rather than a focused check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this whenever asked whether a domain is configured securely, whether it can be spoofed, or to review a domain's email or web security.' This gives clear context, but it does not explicitly mention when to choose the specialized siblings (check_tls, analyse_spf) instead, so it lacks exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_tlsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden of disclosing behavior. It clearly states that the tool performs a read-only inspection and lists the exact data points returned (validity, expiry, issuer, key strength, chain missing intermediates, TLS versions). This gives an agent a precise model of what will happen and what results to expect, with no unstated side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: the first sentence states the purpose and scope with a list of details, and the second sentence gives usage guidance. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description is complete for this simple tool. It explicitly enumerates the return-relevant data (certificate validity/expiry, issuer, key strength, chain, TLS versions) and provides enough context for an agent to decide when and how to invoke it. The single parameter is well-documented in the schema, and the description fills any gaps about the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage: the `domain` parameter is described as 'The domain to check, e.g. example.com. Do not include a scheme or path.' The tool description adds no further parameter semantics, so the baseline score of 3 applies—the schema already provides the necessary meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Inspect') and resource ('domain's TLS configuration'), and enumerates exactly what is inspected (certificate validity, issuer, key strength, chain, TLS versions). This clearly distinguishes it from sibling tools like scan_domain and analyse_spf, which target broader or different concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases ('Use this for certificate problems, expiry checks...') and even mentions a subtle real-world scenario (works in browsers but fails in curl). It lacks explicit alternatives or when-not-to-use instructions, but the use-case list is sufficiently clear and targeted, so this is not a major gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.