Server Quality Checklist

Tool Scores

• check_inboxA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must fully disclose behavior. It states the tool returns subjects, bodies, and auto-extracted links, but it does not indicate whether the operation is read-only, what happens if the inbox is empty, error conditions, or any side effects. For a tool with no annotations, this is insufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that front-load the action and the return value. Every word adds value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the basic function and output, it lacks context on prerequisites (e.g., inbox must exist via create_inbox), what happens on empty results, or any error handling. Given the lack of output schema and annotations, additional detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The tool's description adds no additional meaning to the parameters beyond what the schema already provides (e.g., address purpose, wait_seconds default and range). The description focuses on the return value, not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Check a temporary inbox' and specifies what it returns: subjects, bodies, and confirmation links. This distinguishes it from sibling tools like create_inbox or delete_inbox, which are for management, and wait_for_email, which likely just waits without returning details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies using this tool to retrieve new messages from a temporary inbox, but it does not explicitly say when to use it versus alternatives like wait_for_email or verify_email. No prerequisites or exclusions are mentioned, limiting guidance for selecting the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_inboxA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description merely says 'delete' without elaborating on permanence, irreversibility, permissions, or error handling. Minimal behavioral context beyond the action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no extraneous information. Extremely concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple deletion tool with one parameter, the description is adequate. It lacks mention of irreversibility or error behavior, but overall sufficiently guides the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a description for the parameter. The tool description adds 'temporary inbox and its account' but does not enhance parameter meaning beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'delete' and the resource 'temporary inbox and its account', distinguishing it from sibling tools like create_inbox, check_inbox, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use after you're done with email verification', providing clear context for when to use the tool. However, no exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_emailA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must fully disclose behavior. It explains polling, extraction, and GET request, and specifies return value (HTTP status). However, it does not mention behavior on timeout (e.g., what happens if no email arrives), side effects (e.g., marking email as read), or error handling for failed extraction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, no redundant information. Every sentence serves a purpose, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-step, polling, extraction) and lack of annotations, the description is incomplete. It does not cover prerequisites (inbox must exist), failure modes (timeout, missing link), or side effects, which are critical for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds limited value beyond the schema. It clarifies that 'from' and 'subject_contains' are filters and that timeout_seconds controls wait time, but this is already in the schema. The description does not add new parameter-specific guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'polls for a confirmation email, extracts the verification link, and visits it via HTTP GET.' It distinguishes from siblings like wait_for_email by noting it combines multiple steps into one tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for email verification flows by stating it combines wait_for_email, link extraction, and click. However, it does not explicitly state when not to use it or mention prerequisites like requiring the inbox to already exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_inboxA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must fully convey behavior. It discloses that it uses mail.tm with automatic fallback to 1secmail. However, it does not mention whether the inbox is persistent, any rate limits, or what happens if both services are down.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each earning its place: first states the core function, second adds reliability details. It is front-loaded with the key action and avoids unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two optional parameters and no output schema, the description explains what it returns (a real email address) and the services used. It could mention the return format or error handling, but overall it provides sufficient context for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have descriptions in the schema (100% coverage). The description adds value by explaining that prefix appends a timestamp for uniqueness and that name can be used as a friendly identifier in other tools, which is not evident from the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it creates a temporary email inbox and returns a real email address. It distinguishes from siblings like check_inbox and delete_inbox by focusing on creation. It includes specific technical details about services used (mail.tm, 1secmail).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (when you need a temporary inbox) but does not explicitly state when not to use or compare to siblings like verify_email. No exclusions or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_inboxesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses the tool lists active temporary inboxes, but does not mention any behavioral traits like read-only nature or session requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 9 words, front-loaded with essential information, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters or output schema, the description adequately covers the tool's purpose and scope. Could mention return format, but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description adds meaning beyond empty schema by specifying 'active temporary inboxes created in this session'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all active temporary inboxes' with verb and resource, and distinguishes from siblings like check_inbox, create_inbox, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating scope ('in this session'), but lacks explicit guidance on when to use this vs alternatives like check_inbox for individual inboxes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_emailA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description covers key behaviors: polling, retries with backoff, and return of matched message with links. It does not detail side effects (e.g., whether emails are marked read) or failure behavior (timeout result), but is sufficient for a simple polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. Front-loaded with purpose and immediately clarifies advantage over sibling. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 documented parameters and no output schema or annotations, the description covers the main function and output shape ('matching message with extracted links'). It lacks details on polling interval or backoff specifics, but is adequate for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are documented in the input schema with descriptions. The tool description adds no extra meaning or usage hints for the parameters beyond what the schema provides, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it polls an inbox until a matching email arrives, with retries and backoff. It distills the core action and distinguishes from sibling tool check_inbox by promoting itself as a better alternative to manual looping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advocates using this instead of check_inbox in a loop, providing clear context. However, it does not mention when not to use it or alternatives for other tasks like creating inboxes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.