Server Quality Checklist

Tool Scores

• create_mailboxB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full disclosure burden. It does disclose the mutation nature ('Create'), the ephemeral lifecycle ('temporary'), and the return value, but it omits failure modes (e.g., unavailable username/domain), idempotency, and any side effects or provider resource consumption that would help an agent anticipate outcomes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with zero filler; the primary action is front-loaded and the second sentence about the return value earns its place. Nothing could be removed without losing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The input schema is fully self-documenting and all parameters are optional, but annotations and output schema are absent, leaving the description as the sole behavior and return disclosure. It covers the basics but lacks output shape detail, failure behavior, and sibling routing, which are meaningful gaps for a resource-creating tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: domain, provider, and username each have descriptions including defaults and behavior when omitted. The tool description adds no parameter-level meaning, so the baseline of 3 applies per the high-coverage rule.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Create') and resource ('temporary email mailbox') and states the return value ('email address and account details'), making the core purpose clear. It does not explicitly differentiate itself from siblings such as auto_register or list_mailboxes, so it misses the sibling-distinguishing bar for a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives. The description never mentions conditions like 'use this when you need a fresh disposable address' and names no sibling or exclusion, leaving the agent to infer selection from the tool name and the word 'temporary' alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_domainsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It communicates a read-only 'Get' operation, but it does not disclose whether the provider parameter is optional, what the response format looks like, whether domains are filtered, or whether any network-dependent behavior or failure modes exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no superfluous wording. Key information—action, resource, and scope—is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list-retrieval tool with one optional parameter and no output schema, the description conveys the essential behavior. It could be slightly stronger by noting the default provider or the expected return shape, but these are relatively minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and already documents the provider parameter and its default value. The description's phrase 'for a given provider' loosely maps to the parameter but adds no semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Get'), a clear resource ('list of available email domains'), and a scope ('for a given provider'). It is easily distinguishable from sibling tools like list_providers, which returns providers rather than domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or related steps such as choosing a provider first or using the returned domain with create_mailbox. Usage context is only implied by the tool name and sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messagesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It communicates a read-only list operation and states the return fields, which is meaningful. However, it does not disclose side effects, pagination, empty-result behavior, or how it handles invalid addresses.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with the action and resource front-loaded, followed by the return format. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter list tool with a clear output description, the definition is mostly complete. It lacks explicit sibling differentiation and minor edge-behavior details, but those are not critical for basic invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter address is already fully described in the schema, and the tool description adds no extra meaning beyond implying it identifies the mailbox to check. Baseline 3 is appropriate because schema coverage is 100%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear operation: 'Get the list of received emails' for a temporary mailbox, and specifies the output content (message summaries with ID, from, subject, date). It is distinguishable from the sibling get_message by the word 'list' and 'summaries', though it does not explicitly name or contrast that sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance about when to use get_messages versus siblings like get_message, wait_for_email, or get_verification_code. The only usage signal is implicit: this tool returns summaries rather than full message details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure. 'Get' correctly signals a non-mutating read operation, and the description specifies the return payload (body text, HTML, attachments metadata). It does not disclose error behavior, rate limits, or authorization needs, but for a simple retrieve-by-ID tool this is a limited gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 16-word sentence that front-loads the action and resource, then lists the return components. Every word earns its place, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two required parameters, no nested objects, and no output schema, the description adequately conveys what will be returned. It does not cover error cases or explicitly state that messageId comes from get_messages, but the schema addresses the latter. Slightly more context in the description would be beneficial, but it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both address and messageId already have clear descriptions in the input schema. The tool description itself adds no parameter-level detail beyond the resource being retrieved. Baseline 3 applies because the schema does the heavy lifting for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get the full content of a specific email message', a specific verb and resource, and enumerates the return components: body text, HTML, attachments metadata. The word 'specific' and 'full content' clearly distinguish it from the sibling get_messages tool without needing to reference it directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for retrieving a single message after it has been identified, but it does not explicitly state when to prefer it over get_messages or wait_for_email. It does not mention that a messageId must first be obtained from get_messages, though the input schema does provide this context. The usage guidance is adequate but relies on inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_verification_codeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It explains the core behavior of waiting and extracting, but it does not mention timeout behavior, what happens if no verification code is found, how filters are applied, or any side effects like polling or cleanup. This is a meaningful gap for a tool that blocks until an email arrives.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with no filler. The primary action and result are front-loaded, and the relationship to wait_for_email is explained in one clause. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description tells the agent what the tool returns (extracted code) and its purpose, but it leaves out failure behavior and timeout semantics despite no output schema existing. An agent cannot anticipate errors like a missing code or expired timeout, which are common for this kind of tool. The high schema coverage covers parameters but not behavioral edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies. The schema already explains address, timeoutMs, fromContains, and subjectContains. The description adds context that the email is a verification/confirmation email and that extraction targets the code, which slightly enriches why those filters matter, but it does not add detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: wait for a verification/confirmation email and extract the code. It also explicitly names its relationship to the sibling 'wait_for_email' by calling itself a convenience tool that combines wait_for_email with code extraction, clearly distinguishing it from other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use this tool: whenever a verification code is needed from an email, as it wraps wait_for_email plus extraction. It implies this is preferable for that use case, but it does not explicitly state when not to use it (e.g., if raw email content is needed, use get_message).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_emailA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses blocking/polling behavior and that it returns the first matching message. However, with no annotations provided, it omits important behavioral details such as what happens on timeout and whether the returned email is consumed or remains in the mailbox.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with the mechanism front-loaded, followed by the return behavior and a concrete use case. There is no filler; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for calling the tool given full schema coverage and a clear purpose. However, since there is no output schema and no annotations, the timeout/failure behavior and the structure of the returned message are left unspecified, creating a meaningful gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters are fully documented in the input schema (100% coverage), so the baseline is 3. The description adds no parameter-specific meaning beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Block'), a resource ('matching email' in the mailbox), and the return value ('first matching message'). This clearly distinguishes it from sibling retrieval tools like get_messages, which list existing messages rather than waiting for a new one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Useful for waiting for verification emails during signups.' It implies when to use the tool but does not explicitly mention alternatives or when not to use it, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_providersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. The word 'List' implies a read-only operation and 'all available' signals the complete result set, but the description does not disclose the return format, how capabilities are represented, or any pagination or performance characteristics. It is adequate for a simple listing tool, but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Every word adds meaning: the action, the resource, the scope, and the included capabilities are all communicated efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema listing tool, this description is largely complete: it states what is listed and the kind of information returned. It could be slightly stronger by naming example capabilities, but the lack of complexity and parameters makes the current description sufficient for an agent to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics to explain beyond the empty schema. The baseline of 4 applies because there is nothing for the description to add about parameters, and the description appropriately avoids inventing any.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('all available temporary email providers') and adds the key detail that capabilities are included. This clearly distinguishes it from sibling tools like get_domains or list_mailboxes, which focus on different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an agent needs to enumerate available providers before creating a mailbox or choosing a provider, but it does not explicitly state when to use this tool versus alternatives. There are no exclusions or named sibling tools for comparison, so the usage guidance remains implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• auto_registerA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that an account is created externally, that the resulting key/token is auto-registered in the current session, and that it is returned for the user to save. It does not detail failure behavior, but the core side effects are clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. It front-loads the main purpose, then adds the key behavioral detail about HTTP-only calls and session registration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose, supported providers, and outcome. However, there is no output schema and the description only vaguely says the key/token is 'returned' without describing the return shape or error/timeout behavior, which leaves a meaningful gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are fully documented there. The description adds minimal extra semantic value beyond naming supported providers and the overall outcome, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (register an account), the resource (a provider account), the output (API key/token), and the method (pure HTTP API calls). This distinguishes it from sibling tools that create mailboxes or retrieve messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it is for automatic provider registration without browser automation, and it lists the supported providers. It does not explicitly name alternative tools or exclude cases, but the use case is well implied by the purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_mailboxA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses the destructive effect (deletes the mailbox and all its messages) and adds the provider-support caveat. It doesn't mention irreversibility, permissions, or what happens when a provider doesn't support it, but it covers the core risk.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with the core action first, followed by the scope and a qualifying caveat. Every word earns its place; no padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter destructive operation, the description covers what is deleted, the parameter, and a conditional limitation. Without an output schema or annotations, it might have noted provider-detection or error behavior, but the essential guidance for a correct call is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter, address, is already described as 'The temporary email address to delete.' The description adds no further parameter detail, so it meets the baseline but doesn't enrich beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Delete') and resource ('temporary mailbox') and adds the scope of the action (messages). It is clearly distinct from all siblings, none of which are deletions, so an agent can select it unambiguously.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the use case apparent: clean up a temporary mailbox and its messages. It doesn't explicitly say 'use when done with the mailbox' or list alternatives, but no sibling provides a deletion operation, so the when-to-use context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mailboxesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does add value by clarifying the session-scoped nature of the results and implicitly signaling a read-only operation via 'List'. However, it does not disclose return format, what fields each mailbox entry contains, or how 'session' is defined — gaps that matter for downstream chaining to tools like get_messages.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 9-word sentence that front-loads the verb and resource. Every word earns its place, with no filler, abbreviations, or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no nested objects, no output schema), the description captures the essential behavior completely. Mentioning the return shape (e.g., addresses or IDs) would be helpful for tool chaining but is not critical for such a simple list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the baseline for 0-parameter tools is 4. The description correctly implies the tool takes no input, and there is nothing further it needs to explain about arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a specific resource ('temporary mailboxes'), and a scope qualifier ('created in this session'). It is easily distinguishable from sibling tools: list_providers targets providers, create_mailbox/delete_mailbox are mutations, and get_messages/get_message target messages rather than mailboxes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'created in this session' qualifier provides clear usage context — the agent learns that only session-scoped mailboxes are returned and that persistent or previously-created mailboxes are excluded. However, it does not explicitly name alternatives or state when-not-to-use in the way a 5 would require.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.