Server Quality Checklist

Tool Scores

• mail_search_emailsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses parallel search, deduplication, sorting, exclusion of system mailboxes, and error behavior (timeouts, no results). Annotations already mark it as read-only and idempotent; description adds context without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for args, returns, examples, and error handling. It is somewhat lengthy but efficiently organized and front-loaded with the critical date strategy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, all parameters, usage strategy, error handling, and return format comprehensively. Given an output schema exists and the description is thorough, it provides complete guidance for correct tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has detailed descriptions for each parameter (high coverage), but the description adds extra value with usage examples and strategic advice (e.g., start with since_days=7). Baseline 3 is exceeded due to this added context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search Apple Mail for emails' with specific verbs and resources. It distinguishes from sibling tools like mail_read_email (which reads a single email) and mail_list_mailboxes (which lists mailboxes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides extensive usage guidance including a step-by-step date range strategy, examples mapping common queries to parameters, and instructions for paging. However, it does not explicitly compare to mail_read_email or mail_list_mailboxes, though it implies usage of email_id for reading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mail_list_mailboxesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and destructiveHint, but the description strongly reinforces 'Strictly read-only — no emails are modified and no network calls are made.' It also details error handling (Mail.app unreachable, no accounts), exceeding the annotations' context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with summary, parameters, return format, examples, and error handling. It could be slightly shorter but every section adds unique value; no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple listing nature, annotations, and output schema, the description completely covers purpose, behavior, parameters, return format with both Markdown and JSON examples, and error handling. Additional context (AppleScript, performance hints) makes it fully actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description provides full parameter details: 'include_counts' includes performance trade-off advice ('10k-message mailbox scans in ~10-20s'), and 'response_format' explains default vs. JSON. This adds significant meaning beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists all mailboxes/folders in Apple Mail, grouped by account. It uses specific verbs ('list') and resource ('mailboxes/folders'), and the distinction from sibling tools is clear since siblings handle reading and searching specific emails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Examples show when to use the tool ('What mailboxes do I have?') and when to switch format. However, no explicit when-not-to-use or direct comparison to alternatives is given, though it's implied by the distinct purposes of siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mail_read_emailA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and idempotentHint=true. The description adds that the message read-status is NOT changed, provides performance caveats for large mailboxes, and explains error conditions, all beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns, examples, error handling, notes). It is thorough yet concise, with no unnecessary content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with detailed input schema and annotations, the description is complete: it covers purpose, parameters, return format with examples, error handling, and performance notes. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has detailed descriptions for both parameters, so schema coverage is high. The description reiterates these but adds value through examples of return formats and usage context, justifying a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads the full content of a specific Apple Mail email by its ID, using the verb 'Read' and specifying the resource. It distinguishes from sibling tools by noting that mail_search_emails provides the ID needed for this tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: use after searching with mail_search_emails, and not without an email_id. It includes examples and a clear 'Don't use when' section, which helps the agent decide correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.