Server Quality Checklist

Tool Scores

• save_attachmentC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already disclose destructiveHint=true and readOnlyHint=false, so the agent knows it's a mutating, destructive operation. The description adds no additional behavioral context such as whether files are overwritten, permissions required, or side effects. It merely repeats the obvious 'save' action without extra transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear, front-loaded sentence with no wasted words. It is appropriately brief, though it sacrifices necessary detail. The structure is clean, but the brevity contributes to the lack of context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that writes a file (with destructiveHint and an output schema), the description is too sparse. It omits any mention of return values (though output schema may cover that), overwriting behavior, or prerequisites. Given the moderate complexity (3 parameters) and the availability of annotations and output schema, the description should provide more operational context but does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters but does not. It vaguely references 'one attachment from a message' but fails to mention filename, message_id, or destination explicitly. This leaves the agent without crucial details about what these parameters mean or how they relate to the action.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (save) and the resource (one attachment from a message) with a destination (local folder). It distinguishes from siblings like list_attachments and get_message by focusing on saving. However, it does not explicitly mention that the attachment is identified by filename and message_id, so it's not fully specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as list_attachments or open_in_mail. The description is a bare statement of functionality without any context, exclusions, or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mailbox_statsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint: true, which indicates the tool is safe and read-only. The description adds that it shows 'volume, unread counts and top senders', which conveys what data is returned but does not disclose details like whether it aggregates across all accounts or only the default account, or whether 'recent period' is defined by the 'days' parameter. Since annotations cover the safety profile, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and front-loaded. It directly states what the tool does without unnecessary words, making it easily scannable by an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (not shown here), which likely explains return values. The description is somewhat complete given it's a stats tool, but it lacks explicit guidance on usage context, such as how to use the 'days' parameter or what 'volume' means. With two parameters and no schema descriptions, the description does not provide enough details to fully understand the tool's behavior, though the output schema might help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'recent period' which likely corresponds to the 'days' parameter, and it implies account-specific stats which ties to the 'account' parameter. However, it does not explain the meaning or format of those parameters beyond what the names and defaults suggest. The description only partially compensates for the lack of schema descriptions, so a 3 is reasonable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Show volume, unread counts and top senders for the recent period' clearly states the tool's purpose: it provides a summary of mailbox statistics, including volume, unread counts, and top senders. It is distinguishable from siblings like search_mail or get_message as it is about aggregated stats, not individual messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not specify when to use this tool versus alternatives, nor does it mention any prerequisites or exclusions. It implies usage for summarizing mailbox activity, but no guidance is provided about when to prefer it over other tools like search_mail or get_thread. There is no mention of which account or time period it applies to beyond the parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds a critical security warning about untrusted content inside a fence. This goes beyond the annotations and is vital behavioral context. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first defines the tool's core purpose, the second delivers an important safety warning. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple read tool with output schema and annotations, but it misses parameter explanations for include_body and fetch_if_partial, which are not self-evident. A short note about these parameters would make it more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains message_id (numeric id from search_mail) but does not clarify include_body or fetch_if_partial. With 0% schema description coverage, the description fails to compensate for the ambiguous boolean parameters, leaving their behavior undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (read), the resource (one message), the source (search_mail), and the required identifier (numeric id). It distinguishes itself from siblings like get_thread and search_mail by specifying it reads a single full message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: it reads a message by ID from search_mail, presumably after a search. However, it does not explicitly state when to use this versus alternatives like get_thread, nor does it mention any exclusions or prerequisites. The guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_sendA
  3.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states 'Set save_as_draft=True to put it in Drafts instead of sending,' which is a write operation, directly contradicting the annotation readOnlyHint=true. This is an annotation contradiction, so transparency score is 1.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the purpose. Every sentence adds value, covering non-sending, the one-time token, approval flow, from_account constraint, and draft option. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lays out the full workflow (preview, approval, commit_action), clarifies that nothing is sent until commit, and explains the draft option. An output schema exists, so return values are covered. However, the contradiction with readOnlyHint causes ambiguity about side effects, preventing a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the burden. It adds meaning for from_account (must be from list_accounts) and save_as_draft (draft behavior), but provides no additional semantics for to, cc, bcc, subject, body, or attachments, which are standard email fields. It partially compensates but leaves gaps for many parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that it 'Prepare a new email and return a preview for the user to approve.' It uses a specific verb and resource, and explicitly contrasts with sending by saying 'This does NOT send anything.' This distinguishes it from sibling tools like prepare_reply, prepare_forward, and commit_action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit workflow guidance: show the preview to the user, and only call commit_action once they have approved it. It also specifies that from_account must name one of the accounts from list_accounts, and explains the save_as_draft option. This clearly indicates when and how to use the tool relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_indexA
  3.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation Contradiction: annotations declare readOnlyHint=true, implying the tool does not modify state, but the description says it 'Re-syncs' and 'rebuilds everything from scratch,' which implies changing the local search index. The description also does not disclose side effects or impact of a full rebuild.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and includes usage trigger, duration, and parameter behavior without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-optional-parameter tool, the description covers purpose, trigger, expected duration, and parameter effect. It is mostly complete, but the readOnlyHint contradiction leaves ambiguity about whether the tool has side effects, which prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, full, has no description in the schema, but the description provides meaningful semantics: 'full=True rebuilds everything from scratch' and implies the default is a quick incremental refresh. This compensates for the 0% schema coverage, though it does not explicitly describe false behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Re-sync the local search index with Apple Mail.' This clearly distinguishes the tool from sibling mail/search tools and explains its maintenance role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Run this if a very recent message is missing from search results' gives a clear, concrete trigger condition. It does not explicitly mention when not to use it or name alternatives, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_threadA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and openWorldHint, so the bar is lower. The description adds value by stating the output ordering ('in order') and the dependency on get_message, offering behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded and contains no filler. Every part adds meaning, balancing brevity with key context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with an output schema and read-only annotation, so the description does not need to explain return values. However, it omits how 'limit' behaves and contradicts the 'whole conversation' claim, leaving a gap for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It clarifies that thread_id comes from get_message, which is helpful, but it does not explain the 'limit' parameter. Worse, the claim of 'whole conversation' conflicts with limit's default of 25, which may truncate the thread.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read'), identifies the resource ('whole conversation'), notes ordering ('in order'), and provides a source for the thread_id ('from get_message'). This clearly distinguishes it from sibling tools like get_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states the prerequisite and context: you need a thread id from get_message. This gives a clear 'when to use' signal, though it does not explicitly mention when not to use the tool or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_mailA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds useful behavioral context beyond annotations: it searches all accounts at once, includes body text, matches subject/sender/recipients/body, and returns numeric message IDs for downstream tools. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the main purpose in the first sentence. Each subsequent sentence adds distinct value: query syntax, empty-query behavior, time filter, and return-value usage. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core search behavior and downstream ID usage, and an output schema exists to explain return values. However, it omits important context for a 10-parameter tool: how limit/offset pagination works, what the default days=0 means, and how the sender/account/mailbox/filter parameters interact with query.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It explains query and days well, but the other eight parameters (limit, offset, sender, account, mailbox, unread_only, flagged_only, with_attachments) receive no explanation beyond their schema names, leaving significant ambiguity for an agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search mail across every account at once, including message bodies.' It clearly distinguishes search_mail from sibling retrieval tools like get_message and get_thread by emphasizing cross-account, body-inclusive search and by stating it returns IDs for use with those tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical usage guidance: query semantics, exact-phrase quoting, empty-query browsing, and the days=7 time filter. It implies the workflow of searching first then using get_message/get_thread/prepare_reply/prepare_forward, though it does not explicitly state when not to use this tool or name alternatives for other mail operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_forwardA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, so the description adds value by explaining that the tool does not send and returns a preview for approval. This gives context beyond the annotations about the intended workflow and non-mutating nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences) and front-loaded: the first sentence states the core purpose, and the second provides a crucial qualification (does not send) plus an explicit next step. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema (preview) and the sibling tools include commit_action, the description covers the key aspects: it generates a preview, does not send, and leads to a commit. It omits details about parameter usage, but that could be inferred from schema and sibling context. Still, for a 5-parameter tool, a bit more context on when to use it (e.g., in a forward workflow) would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should have compensated by explaining parameter meanings. It doesn't address any of the 5 parameters (to, note, message_id, from_account, save_as_draft). While some names are self-explanatory, the description adds no additional semantics or usage hints for parameters, leaving agents to guess at domain-specific conventions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Prepare a forward of a message and return a preview for approval.' It uses a specific verb ('prepare') and resource ('forward of a message'), and distinguishes itself from siblings like prepare_send and prepare_reply by explicitly mentioning 'forward'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: it notes that the tool does not send and instructs to use commit_action after user approval. This sets expectations for the preparation step. However, it doesn't explicitly name alternatives like prepare_send or prepare_reply, though the tool name itself implies the forward-specific case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_replyA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds useful behavioral details: it quotes the original message, replies from the account that received it, and does not send. These traits go beyond the annotation and help the agent understand expected side effects and flow. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the primary purpose and followed by key behavioral notes and a pointer to commit_action. Every sentence carries meaningful information without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (so return values are covered), the description adequately covers the tool's core behavior: it prepares a reply, includes quoting and account selection, and explicitly states it does not send. While parameter details are not explained, the tool is a preparatory action and the description covers the essential workflow. Minor gaps like the effect of save_as_draft or reply_all remain but are secondary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description does not explain any of the four parameters (body, message_id, reply_all, save_as_draft). It only implicitly references body via 'Quotes the original beneath your text' but does not clarify the meaning or interaction of reply_all and save_as_draft. The description fails to compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'Prepare a reply to a message and return a preview for approval.' It clearly distinguishes from siblings (prepare_send, prepare_forward) by focusing on replies, including the quoting behavior and account selection. The final clause 'Does not send; use commit_action after the user approves' further clarifies its role as a preparatory step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it is for replying to a message, not sending. It explicitly directs the user to use commit_action after approval, indicating the follow-up step. However, it does not explicitly name alternative tools (e.g., prepare_send, prepare_forward) or state when not to use it, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pendingA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is consistent with the annotations: readOnlyHint is true and the description says 'Show actions,' indicating no side effects. While it doesn't add extra context like 'does not modify anything,' the annotation already covers this, so no contradiction and adequate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single sentence that conveys the essential purpose without any fluff or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there are no parameters and the output schema is present (though not detailed), the description sufficiently covers what the tool does. It does not need to explain return values because the output schema exists. The simplicity of the tool makes the description complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100%. There is nothing to describe, and the description correctly omits any parameter details, making it fully aligned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Show actions that are prepared and awaiting approval.' It uses a specific verb ('Show') and resource ('actions that are prepared and awaiting approval'), which distinguishes it from siblings like prepare_send and commit_action. It is clear and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (e.g., when you want to review pending actions), but it does not explicitly state when to use this tool versus alternatives. Given the sibling context, it is obvious, but the description alone lacks explicit 'use when' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_attachmentsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already provide readOnlyHint=true, so the read-only nature is conveyed. The description adds that it returns names, types, and sizes, but this is likely captured in the output schema. No additional behavioral details are given, and no contradiction exists. Since annotations cover the safety profile, a score of 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the action and purpose. There is no wasted information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one parameter and an output schema present, the description fully conveys the tool's purpose. Additional details about return structure are presumably in the output schema, so the description is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the tool only has one parameter (message_id), which is self-explanatory. The description's reference to 'a message' clarifies the parameter's meaning. Given the simplicity and context provided, the description compensates adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource as 'attachments on a message', distinguishing it from siblings like save_attachment (which saves) and get_message (which retrieves the message itself).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when to use this tool (to list attachments for a given message), but does not provide explicit exclusions or mention of alternative tools for similar tasks. It gives clear context without negation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  4.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description does not contradict this. However, it adds no further behavioral details beyond the annotation, such as side effects or access constraints, so it stays at the baseline for annotated tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no redundancy. It delivers the core purpose and usage guidance directly, making it easy for an agent to parse and apply.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there are no parameters and the output schema is not depicted, the description adequately covers the tool's function. It provides sufficient context for an agent to know when and why to call it, without needing to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema fully covers this. The description adds no unnecessary parameter details, and there is nothing missing when the agent needs to invoke the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all mail accounts configured in Apple Mail, with a specific verb and resource. It distinguishes itself from sibling tools that perform other actions like sending or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs when to use the tool: 'Call this first when you need an account name for sending, or when the user refers to an account loosely.' This gives clear context and preempts ambiguity in user requests.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• commit_actionA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false, so the description's claim that it's the only mutating tool is consistent. It adds extra context about the token's single-use and expiration, which is valuable beyond the annotations. However, it doesn't elaborate on the specifics of the mutation, but that's acceptable given annotations cover the core safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it opens with the purpose, then immediately gives the critical timing and token constraints, and ends with a strong differentiator. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single token param), the description covers all essential aspects: what it does, when to use it, token semantics, and its unique role. An output schema is available, so return format isn't needed. This is fully adequate for effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides minimal info for the token parameter (just 'Token'). The description enriches it by stating the token is single-use and expires, implying it should come from a prepare_* tool. This adds meaningful semantic context despite the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to carry out an action that was previously prepared and shown to the user. It also differentiates from sibling tools by noting it's the only one that changes anything, making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly specifies when to call it ('Only call this after the user has seen the preview from a prepare_* tool and explicitly approved it') and provides critical constraints (token is single-use and expires). This is clear, actionable guidance with no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_in_mailA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds valuable behavioral context beyond annotations: whole_thread=True opens each message in its own window rather than a threaded view, and 'Changes nothing.' consistently reinforces the readOnlyHint=true annotation. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose first, then usage conditions, then parameter nuance and an alternative. Every sentence earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only UI tool with an output schema and strong annotations, the description covers trigger conditions, parameter intent, side effects, and alternatives. It is complete enough for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so thoroughly for whole_thread, explaining the 'own window' behavior and caution against overuse. message_id is implied by the tool purpose but not explicitly detailed; still, the meaning is clear enough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb + resource: 'Open a message in Apple Mail on screen.' It also distinguishes from siblings by explaining that chat clients strip message:// links, making this the reliable path, and by referencing get_thread for reading in chat.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use conditions ('when the user asks to open, show or pull up a message'), explicit when-not-to-use guidance for whole_thread, and a direct alternative ('use get_thread' to read a conversation in chat). This is clear, actionable differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.