Server Quality Checklist

Tool Scores

• check_domainA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, openWorldHint), the description details the fail-open behavior, the meaning of response fields (disposable, action, check flags), and optional signals (mx_valid, relay, etc.). It also clarifies that a failed check returns checked:false with action:allow and a note, which is critical behavioral context not in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but each sentence provides necessary context for a nuanced tool. It is well structured with clear concepts and edge-case disclosures (fail-open, opt-in signals). Slightly verbose but justified; no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description carries the full burden of explaining return values and behavior. It thoroughly covers the core verdict, action handling, optional fields, and failure semantics. Complete for a single-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already explains that domain should be a bare domain without @ sign. The description reiterates this and adds a usage example, but does not add new syntax or format details beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Checks a single bare domain... to see whether it is disposable.' It also distinguishes from sibling tools by emphasizing 'single' and 'bare domain (no local part)', which separates it from check_email and check_batch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives usage context: 'for example when only a company's domain is known and not a specific address.' It implies use cases but does not explicitly name alternatives or exclusions. Clear enough for an agent to select it over the email/batch variants.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_emailA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint=true and openWorldHint=true, but the description goes much further by disclosing the fail-open behavior: on network problems, timeouts, rate limits, or server errors, the tool still succeeds with `checked:false` and `action:allow`, and this must be treated as inconclusive. It also details optional response fields (`mx_valid`, `relay`, etc.), which is valuable context not present in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but it is well-structured and front-loaded with the core action. It then explains the disposable concept, response fields, and failure mode—each element adds value, especially given there is no output schema. It is slightly verbose but not redundant; every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter and no output schema, the description fully equips an agent to invoke the tool and interpret results. It covers input, all relevant output fields, the meaning of the verdict, and the critical fail-open behavior that could otherwise mislead the agent. The description is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single `email` parameter, which already includes a clear example ('someone@example.com'). The description repeats that example but adds no new technical constraints or formatting rules. It focuses on the semantic meaning of 'disposable,' which is helpful for concept understanding but does not deepen parameter semantics beyond the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource: 'Checks a single email address to see whether it is disposable.' This specifically distinguishes it from sibling check_batch (which would handle multiple addresses) and check_domain (which targets domains), and the scope is immediately evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly scopes the tool to 'a single email address,' which implies that batch checks should use the check_batch sibling. While it does not name alternatives or exclusion criteria, the clear single-address context provides sufficient guidance for selecting between the listed sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_batchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only say readOnly and openWorld. The description adds crucial behavioral details: fail-open behavior (call succeeds with checked false, action allow, note), the meaning of disposable, and the purpose of extra signals like mx_valid, relay, public_domain, and did_you_mean. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized for a complex batch tool. Every sentence adds value: efficiency, input format, definition of disposable, response fields, and fail-open handling. Well-structured and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description fully covers return values and edge cases. It explains all relevant response fields, the meaning of a disposable address, and the inconclusive fail-open outcome. This is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the 'items' parameter with 100% coverage, including the @-sign rule and min/max. The description mainly echoes this rule without adding new parameter-specific semantics, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool checks up to 100 emails or domains in a single call, with a specific verb and resource. It explicitly distinguishes from sibling tools by noting efficiency over looping single-item tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use this tool instead of alternatives (batch vs loop), how to structure the mixed list, and how to interpret the response. Includes fail-open behavior, making it clear what to expect on errors.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.