Server Quality Checklist

Tool Scores

• inbox_check_deleteB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates that the operation is destructive ('Delete') and irreversible, which is critical for a mutation tool. However, it lacks details on permissions, error handling, or response format, leaving gaps in behavioral understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of only two sentences that directly state the action and a critical warning ('Irreversible'). Every word earns its place, with no wasted information, making it efficient for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature, no annotations, no output schema, and low schema coverage, the description is incomplete. It covers the irreversible aspect but misses key details like parameter meaning, error scenarios, and what happens upon success, making it inadequate for safe and effective use by an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 1 parameter with 0% description coverage, so the description must compensate. It does not mention the 'token' parameter at all, failing to explain its purpose (e.g., authentication or test identifier). This leaves the parameter undocumented, reducing the tool's usability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and the resource ('a test and all its results / screenshots'), which is specific and unambiguous. However, it does not explicitly differentiate this tool from its siblings (e.g., 'inbox_check_create', 'inbox_check_list'), which would require mentioning it's for deletion versus creation or listing, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'inbox_check_create' or 'inbox_check_list'. It mentions the action is 'irreversible', which hints at caution but does not specify prerequisites, conditions, or explicit alternatives, leaving the agent with minimal usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inbox_check_statusB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what information is returned (placement status, SPF/DKIM/DMARC results, stats, screenshot URLs) which is helpful context beyond just being a read operation. However, it doesn't mention potential limitations like rate limits, authentication requirements, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured as a single sentence that clearly communicates what the tool returns. Every element (placement status, authentication checks, stats, screenshot URLs) earns its place by specifying the scope of information retrieved.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read operation with no annotations and no output schema, the description provides adequate information about what data is returned. However, it could be more complete by mentioning the format of returned data or any prerequisites beyond having a valid token.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single 'token' parameter. The description adds that this token comes from 'inbox_check_create', which provides useful context about parameter origin, but doesn't add significant semantic meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the current state of a test' with specific details about what information is retrieved (placement status, authentication results, stats, screenshot URLs). It uses a specific verb ('Get') and identifies the resource ('test'), but doesn't explicitly differentiate from siblings like 'inbox_check_list' or 'inbox_check_me'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the token parameter comes from 'inbox_check_create', which implies a workflow sequence, but doesn't state when to choose this over other sibling tools like 'inbox_check_list' or 'inbox_check_me' for checking test status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inbox_check_listA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses pagination behavior ('Supports cursor pagination via created_at') and ownership scope ('owned by this API key'), which are useful. However, it doesn't mention rate limits, authentication needs, or what happens on errors, which are gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by a key behavioral detail (pagination). Every word earns its place with no redundancy or fluff, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and pagination but lacks details on return values, error handling, or full behavioral context. For a list tool with 3 parameters, it's adequate but has clear gaps in completeness, especially without structured output information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (limit, cursor, status) with descriptions and constraints. The description adds no additional parameter semantics beyond what's in the schema, such as explaining how 'cursor' works with 'created_at' or default values, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('recent tests owned by this API key'), specifies ordering ('most recent first'), and distinguishes from siblings by focusing on listing owned tests rather than creating, deleting, checking status, or other operations. It's specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing owned tests with pagination, but doesn't explicitly state when to use this tool versus alternatives like inbox_check_status or inbox_check_me. No exclusions or prerequisites are mentioned, leaving some ambiguity about the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inbox_check_meA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a read-only operation ('Return') that retrieves metadata, but does not mention potential limitations like rate limits, authentication requirements, or error conditions. The description adds basic behavioral context but lacks depth for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently communicates the tool's purpose and scope. Every element ('Return', 'API key's metadata', specific metadata items) earns its place with zero wasted words, making it front-loaded and immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is reasonably complete for a metadata retrieval operation. However, without annotations or output schema, it could benefit from mentioning the return format or any authentication requirements. It covers the core purpose adequately but has room for more contextual detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, maintaining focus on the tool's purpose without unnecessary parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Return') and resource ('this API key's metadata') with detailed scope ('tier, enabled features, allowed providers, current daily/monthly usage vs. limits'). It distinguishes itself from siblings like 'create', 'delete', 'list', and 'status' by focusing on metadata retrieval rather than management operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context for checking API key metadata, but does not explicitly state when to use this tool versus alternatives like 'inbox_check_status' or 'inbox_check_list'. It provides clear context about what information is returned, but lacks explicit exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inbox_check_createA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: it's a creation operation (implies mutation), discloses quota consumption ('each successful create consumes one unit'), and specifies output behavior ('Returns a token and the seed addresses'). It could improve by mentioning error handling or permissions, but covers essential aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, with three sentences that each add value: the core action, output details, and quota information. There is no wasted text, and it efficiently communicates necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (creation operation with quota constraints) and lack of annotations or output schema, the description is mostly complete. It covers purpose, output, and behavioral traits like quota usage. However, it could be more complete by detailing error cases or response formats, but it adequately supports agent usage in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what the schema provides, such as clarifying usage or constraints. This meets the baseline for high schema coverage, but does not enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create an inbox-placement test') and resource ('inbox-placement test'), distinguishing it from sibling tools like delete, list, me, and status operations. It provides concrete output details ('Returns a token and the seed addresses you must send your test email to'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by mentioning API key quotas ('daily/monthly quota; each successful create consumes one unit'), which helps determine when to use this tool. However, it lacks explicit guidance on when to use this versus alternatives like inbox_check_list or inbox_check_status, and does not specify prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.