Server Quality Checklist

Tool Scores

• validate_recipientsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate a read-only, idempotent, non-destructive operation. The description adds valuable behavioral guarantees beyond that by stating 'Nothing is sent and no connection is made,' which meaningfully reduces the agent's concern about side effects. It also clarifies that the tool provides refusal reasons, not just a boolean result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the first states the core behavior, the second emphasizes side-effect safety, and the third gives usage guidance. The most important functional information is front-loaded, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single well-documented parameter and an output schema, this description is complete. It covers what the tool does, what it guarantees about side effects, and when it should be used. There are no significant gaps that would prevent an agent from selecting and invoking it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the addresses parameter is already fully documented in the schema. The description refers to 'given addresses' but does not add additional semantic detail about the parameter format, constraints, or edge cases beyond what the schema provides, warranting the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Says which') and clearly identifies the resource (given addresses the server is allowed to write to) and the outcome (which are allowed and why others are refused). It effectively distinguishes this from the sibling tools, which focus on server info, message preview, and connection testing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent to use this before composing a message and contrasts it with discovering the refusal afterward. It gives clear timing and context, though it does not explicitly name alternative tools or state when not to use it beyond that timing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint=false. The description adds a meaningful behavioral trait beyond those: it performs no network interaction. It also clarifies that it reflects current state such as limits and whether sending is enabled, which helps set expectations without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences with no filler. The first sentence front-loads the resource and its concrete outputs; the second explains when to call the tool and adds a key behavioral property. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter configuration-inspection tool with a rich output schema and safety annotations, the description is complete. It explains what the tool reports, when to call it first, and that it is non-network-touching, which is sufficient for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter burden on the description. Per the 0-param baseline, this dimension earns a 4; the description does not need to explain parameter semantics that do not exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Reports') and clearly names the resource: server configuration. It enumerates exactly what the tool provides — SMTP endpoint, sender address, allowed write targets, current limits, and sending enabled state — which makes its purpose unmistakable and distinct from the network-touching sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance to 'Call this first' and explains why: it answers whether sending is possible and to whom without touching the network. This gives clear usage context, though it does not explicitly name sibling alternatives or state when not to use them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_mailA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds substantial behavior beyond the annotations: the no-network guarantee ('without connecting to anything'), full validation parity with send (allowlist, recipient limit, attachment policy, size limit), and the security-relevant output behavior (attachment payloads summarised by name, size and digest rather than printed). The readOnlyHint and idempotentHint annotations are consistent — no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences, each earning its place: core behavior, validation parity and workflow purpose, then output summarization caveat. The most decision-relevant fact (does not send) is front-loaded. No redundancy with the title or annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a complex 10-parameter tool with an output schema and full annotations. It covers what happens (preview built, checks run, headers/bodies returned), what does not happen (no connection), why to use it (pre-approval), and the attachment output caveat. The output schema covers return values and the rich parameter schema covers inputs, so nothing an agent needs to select and invoke correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema fully documents all 10 parameters. The description adds some contextual reinforcement by naming the checks that constrain parameters (recipient limit, attachment policy, size limit) and the digest summarization that affects how attachments appear in output, but it does not add per-parameter meaning. Baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Builds exactly the message send_mail would build and returns its headers and bodies, without connecting to anything.' This clearly distinguishes it from actually sending (send_mail) and from the siblings (validate_recipients, test_connection), which serve different purposes. The title reinforces the same meaning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use context: 'this is the way to find out whether a message is acceptable before asking a human to approve it.' It also signals that it subsumes validation tasks by stating every send-time check runs here. However, it does not explicitly contrast with the listed siblings or state when-not-to-use, so the routing guidance is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_connectionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable protocol-level context beyond those annotations: TLS negotiation, authentication, connection close, and the explicit guarantee that no message is sent. This clarifies the side-effect profile (network connection but no data mutation) and helps the agent understand what the tool actually does at runtime.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the first states the core action, the second draws a critical boundary (no message sent), and the third gives routing guidance. Zero wasted words; the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with a rich annotation set (read-only, idempotent, non-destructive) and an output schema present, the description covers everything an agent needs: what it does, what it doesn't do, and when to choose it. There are no gaps in prerequisites, side effects, or return expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (nothing to document). Per the baseline for parameterless tools, a 4 is appropriate — the description doesn't need to compensate for any undocumented parameters because there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Opens a connection to the SMTP server, negotiates TLS and authenticates, then closes it again.' It clarifies scope with 'No message is sent,' distinguishing it from mail-related siblings. The configuration-vs-delivery framing further separates it from validate_recipients and preview_mail without needing to open any schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The final sentence gives explicit situational guidance: 'Use it to tell a configuration problem apart from a delivery problem.' This tells the agent when the tool is appropriate. It doesn't explicitly name sibling alternatives or state when not to use it, but the context is clear enough that an agent can infer the routing decision.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.