Server Quality Checklist

Tool Scores

• create_mailboxC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool creates a mailbox but doesn't describe what happens on success/failure, whether it requires specific permissions, if it's idempotent, or what errors might occur. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one short sentence) and front-loaded with the core purpose. Every word earns its place, with no redundant information or fluff. It efficiently communicates the essential action without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool (creating a mailbox) with no annotations and no output schema, the description is incomplete. It doesn't cover behavioral aspects like permissions, idempotency, or error handling, nor does it describe what the tool returns. For a tool that modifies state, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'name' fully documented in the schema as 'Name of the mailbox to create'. The description adds no additional parameter semantics beyond what the schema provides, such as naming constraints or examples. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('new mailbox/folder'), making the purpose immediately understandable. It distinguishes from siblings like 'delete_mailbox' and 'get_mailboxes' by specifying creation rather than deletion or retrieval. However, it doesn't explicitly differentiate from all siblings (e.g., 'auto_organize' could also involve mailbox creation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing proper permissions), when not to use it (e.g., if a mailbox already exists), or direct alternatives among siblings (e.g., using 'get_mailboxes' first to check existence). The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_mailboxC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Delete' implies a destructive mutation, it doesn't specify whether deletion is permanent or reversible, what happens to contained messages, required permissions, or error conditions. This leaves significant gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core functionality without unnecessary words. However, it could be more front-loaded with critical behavioral information given the destructive nature of the operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations and no output schema, the description is insufficient. It doesn't address what happens after deletion, whether confirmation is needed, error handling, or how this interacts with sibling tools. The context demands more completeness for safe agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'name' parameter. The description doesn't add any parameter-specific context beyond what's in the schema (e.g., format constraints, examples, or relationship to other tools). Baseline 3 is appropriate when schema does the documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('an existing mailbox (folder)'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_messages' or 'create_mailbox', which would require explicit comparison to earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'delete_messages' or 'move_messages'. There's no mention of prerequisites (e.g., mailbox must exist), consequences, or typical scenarios for deletion versus other operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action without disclosing critical behavioral traits. It doesn't mention whether deletion is permanent or reversible, permission requirements, error handling, or side effects on the mailbox, which are essential for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words, making it highly efficient and front-loaded. It directly communicates the core function without unnecessary elaboration, earning its place succinctly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral outcomes, error cases, or return values, leaving significant gaps in understanding how the tool operates and what to expect after invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, fully documenting both parameters, so the description adds no additional parameter information beyond what the schema provides. This meets the baseline for high schema coverage, but doesn't compensate with extra context like format examples or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('messages from a mailbox'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'move_messages' or 'delete_mailbox', which would require more specificity about scope or alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'move_messages' or 'delete_mailbox', nor does it mention prerequisites or exclusions. It lacks context about appropriate scenarios, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_attachmentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action but doesn't cover critical aspects like authentication requirements, rate limits, error conditions (e.g., invalid message ID), output format (e.g., file stream vs. metadata), or whether this is a read-only operation. The description is minimal and lacks necessary behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no annotations, no output schema), the description is insufficient. It doesn't explain what the tool returns (e.g., file content, download link), error handling, or operational constraints. For a download operation with multiple inputs, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain relationships between parameters or provide usage examples). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Download') and target ('an attachment from a specific message'), providing a specific verb+resource combination. However, it doesn't differentiate from potential sibling tools like 'get_messages' which might also retrieve message content, though the focus on attachments is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid message ID), exclusions, or how it differs from other retrieval tools like 'get_messages' that might include attachments in their output.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, it doesn't specify whether this requires authentication, what format messages are returned in, whether there are rate limits, or how errors are handled. For a tool with 3 parameters and no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core functionality without unnecessary words. It's appropriately sized for a straightforward retrieval tool and gets directly to the point with zero wasted language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what format messages are returned in, whether there's pagination, what authentication is required, or how to handle errors. Given the complexity of email retrieval and the lack of structured metadata, the description should provide more operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with all parameters well-documented in the schema itself. The description adds no additional parameter information beyond what's already in the schema properties. This meets the baseline expectation when the schema does the heavy lifting, but doesn't provide extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('email messages') with scope ('from specified mailbox'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'search_messages' or 'get_mailboxes', which would require more specific language about what distinguishes this retrieval operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_messages' or 'get_mailboxes'. There's no mention of prerequisites, typical use cases, or exclusions that would help an agent choose appropriately among the available email-related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mark_as_readC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action but doesn't mention permissions required, whether the operation is reversible, error handling, or what happens if invalid message IDs are provided. This is a mutation tool with significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple tool and immediately communicates the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't address behavioral aspects like side effects, error conditions, or what the tool returns. The context signals indicate this is a non-trivial operation requiring more disclosure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any additional meaning about the parameters beyond what's in the schema, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('mark as read') and resource ('email messages'), making the purpose immediately understandable. It doesn't specifically differentiate from sibling tools like 'set_flags' which might also handle read status, but the core action is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'set_flags' or other email management tools. The description only states what it does, not when it's appropriate or what prerequisites might exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• move_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Move messages between mailboxes' implies a mutation operation, but it doesn't disclose critical traits such as whether the move is permanent, if it requires specific permissions, what happens to the source messages (e.g., deletion or copying), or potential side effects like updating message IDs. This leaves significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resources, making it easy to parse quickly. Every word earns its place without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with no annotations and no output schema, the description is insufficiently complete. It lacks details on behavioral traits (e.g., permanence, permissions), error conditions, or what the tool returns. For a tool that modifies data, this leaves the agent with significant uncertainty about how to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for all three parameters (sourceMailbox, destinationMailbox, messageIds). The description doesn't add any meaning beyond what the schema provides, such as explaining mailbox naming conventions or message ID formats. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Move messages between mailboxes' clearly states the action (move) and the resources (messages, mailboxes), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_messages' or 'set_flags' which also manipulate messages, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., mailboxes must exist), exclusions (e.g., cannot move to non-existent mailboxes), or comparisons to siblings like 'delete_messages' or 'set_flags' for different operations on messages.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool searches messages but doesn't describe what happens during execution (e.g., whether it's read-only, if it requires authentication, potential rate limits, or what the output looks like). The phrase 'using various criteria' is vague and doesn't add meaningful behavioral context beyond what the parameters imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a search tool, though it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return values, error conditions, or behavioral traits like whether it's safe to use or has side effects. For a search tool with multiple filtering options, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 7 parameters with clear descriptions. The description adds no specific parameter information beyond 'various criteria,' which doesn't provide additional meaning or context. This meets the baseline of 3 since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('messages'), specifying the action and target. It mentions 'various criteria' which hints at filtering capabilities, but doesn't explicitly differentiate from sibling tools like 'get_messages' or 'auto_organize' that might also retrieve messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer search_messages over get_messages (which might retrieve specific messages without filtering) or auto_organize (which might involve searching as part of organization). No context about prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_emailC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states the action ('Send') but doesn't cover critical aspects like authentication requirements, rate limits, error handling, or whether the email is sent immediately or queued. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for the tool's complexity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens on success/failure, return values, or behavioral constraints. For a tool that sends emails, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (to, subject, html, text) with clear descriptions. The description adds no additional parameter information beyond what's in the schema, meeting the baseline for high coverage but not enhancing understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send') and resource ('email through iCloud Mail'), making the purpose immediately understandable. However, it doesn't distinguish this from potential sibling tools like 'move_messages' or 'set_flags' that might also involve email operations, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication), exclusions, or compare it to sibling tools like 'create_mailbox' or 'delete_messages', leaving the agent with no context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_flagsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does without disclosing behavioral traits. It doesn't mention permission requirements, side effects (e.g., whether flags persist), error handling, or rate limits, which are critical for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose with no wasted words. It uses parentheses to include helpful examples without cluttering the main statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral context, error cases, or return values, leaving significant gaps for the agent to operate safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description adds minimal value by hinting at flag examples (e.g., 'read, unread, flagged'), but doesn't elaborate beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Set') and resource ('flags on messages'), with examples of specific flags like 'read, unread, flagged'. It distinguishes from siblings like 'mark_as_read' by covering multiple flag types, though it doesn't explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'mark_as_read' or 'auto_organize'. The description mentions flag types but doesn't specify scenarios or prerequisites for usage, leaving the agent to infer context from parameter names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• auto_organizeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'automatically organize' and hints at rules, but doesn't describe what 'organize' entails (e.g., moving emails, potential side effects), authentication needs, rate limits, or error handling. For a mutation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality ('Automatically organize emails') and adds essential detail ('based on rules...'). There's zero waste, and it's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, mutation operation, no output schema, and no annotations), the description is minimally adequate. It covers the purpose but lacks behavioral details, usage context, and output expectations. It meets a basic threshold but has clear gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain rule interactions or organizational logic). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Automatically organize emails based on rules (sender, subject keywords, etc.)'. It specifies the verb ('organize'), resource ('emails'), and mechanism ('rules'), but doesn't explicitly differentiate it from sibling tools like 'move_messages' or 'set_flags' that might also organize emails in different ways.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., existing mailboxes), compare to siblings like 'move_messages' for manual operations, or specify use cases like bulk automation versus single actions. Usage is implied but not articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_configB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states what the tool does but doesn't disclose behavioral traits like what 'properly configured' means, what happens if configuration is invalid, whether it's read-only or has side effects, or what the output format might be. For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a zero-parameter tool and gets straight to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and the description's limited behavioral disclosure, this is incomplete for a configuration checking tool. It doesn't explain what 'properly configured' means, what validation criteria are used, or what format the result takes. The description should provide more context about the checking behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the baseline is 4. The description doesn't need to compensate for any parameter documentation gaps since there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: checking if environment variables are properly configured. It uses a specific verb ('check') and resource ('environment variables'), but doesn't differentiate from sibling tools, which are all email-related while this is a configuration tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing considerations, or relationship to sibling tools. The context suggests it's for configuration validation, but no explicit usage context is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_connectionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Test') but doesn't reveal critical traits: whether this is a read-only diagnostic, if it requires authentication, potential side effects (e.g., network traffic), error handling, or output format. This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes relevant technical details (IMAP and SMTP) without fluff, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of testing server connections, the description is incomplete. With no annotations and no output schema, it fails to explain what the test entails, what results to expect (e.g., success/failure indicators), or behavioral aspects like timeouts or retries. This leaves significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add param info, but that's unnecessary here. A baseline of 4 is appropriate as it avoids redundancy while the schema fully handles the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Test the email server connection (IMAP and SMTP).' It specifies the verb ('Test') and the resource ('email server connection'), with additional detail about the protocols involved. However, it doesn't explicitly differentiate from sibling tools like 'check_config', which might have overlapping functionality, preventing a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., after configuration changes), exclusions, or comparisons to siblings like 'check_config', leaving the agent with minimal context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mailboxesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'List all available mailboxes,' which implies a read-only operation, but doesn't specify details like whether this requires authentication, returns paginated results, or includes metadata. For a tool with zero annotation coverage, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('List all available mailboxes') that is front-loaded and wastes no words. It directly conveys the core purpose without unnecessary elaboration, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks context on usage, behavior, or output, which could be improved for better agent guidance. It meets the minimum viable threshold but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning there are no parameters to document. The description doesn't need to add parameter semantics, so it meets the baseline of 4 for tools with no parameters, as it doesn't introduce confusion or omissions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all available mailboxes'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_messages' or 'search_messages' that might also involve mailbox operations, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, such as whether this should be used before other mailbox operations or as a standalone query. This leaves the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.