Server Quality Checklist

Tool Scores

• list_messagesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only declare readOnlyHint, so the description must supply behavioral depth. It adds sorting (newest first), output scope (summaries only), and a performance trait (server-side filter execution). This goes well beyond the annotation without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, no filler, with the core behavior and ordering in the opening line. Every sentence carries a distinct fact: filtering options, no-criteria behavior, server-side performance, and summary-only return. This is appropriately compact for a 12-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no required parameters and a fully described schema, the description covers the behavior that matters: ordering, summary output, body alternative, and paging behavior. It doesn't describe exact summary fields or explicitly distinguish all sibling tools, but the schema and the get_message pointer fill most of the gap. Given no output schema, the 'summaries only' statement is sufficient directional guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already documents all 12 parameters, so the baseline is 3. The description adds a useful grouping of filter types and clarifies that omitting criteria turns the call into a page-through of the mailbox. It also ties filtering to server-side execution, which is operational context the schema does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Lists messages newest first') and enumerates the viable filters, including sender, recipient, subject, body, date range, and flags. It also points to get_message for full body, which helps disambiguate from sibling list/get tools. However, it never explicitly contrasts this with list_new_messages, so sibling differentiation is incomplete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states when to page without criteria and why: 'With no criteria it simply pages through the mailbox' and 'searching a large folder is cheap' due to server-side filtering. It explicitly routes body retrieval to get_message: 'Returns summaries only — use get_message for the body.' It doesn't mention when to prefer list_new_messages over this tool, so exclusions are partial.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_new_messagesA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fully discloses the side-effecting behavior: messages are marked with the AiSeen keyword, while human read/unread state is 'never touched.' It also presents dry_run as a preview mechanism. This goes well beyond the annotations, which only state readOnlyHint=false and destructiveHint=false.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences deliver the core behavior, the side effect, the read/unread distinction, and the dry_run option without any filler. The most important information is front-loaded, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers ordering, side effects, and preview behavior, which is enough for an agent to invoke the tool correctly. It does not describe the return format, but with no output schema the lack of explicit return-structure details is a minor gap given the tool's straightforward nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both limit and dry_run are already clearly documented in the schema. The description adds only a brief reminder to use dry_run for previewing, so it does not materially improve parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Returns messages this server has not handed over yet, newest first.' It also explains the distinctive AiSeen-marking behavior, clearly separating this tool from the sibling list_messages and the human read/unread state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a polling use case by explaining that marking ensures 'the next call returns only what arrived since,' and it gives a clear dry_run usage instruction. However, it never explicitly states when to choose this tool over alternatives like list_messages, nor does it name exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_infoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true, and the description consistently describes a read-only reporting behavior ('Reports what this account and server can do'). It adds useful behavioral context by framing the tool as a diagnostic starting point and listing the categories of information returned, which goes beyond merely restating the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states the core purpose and enumerates specific report contents, and the second sentence provides a practical usage cue. There is no filler or repetition of the tool name or schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete enough given the tool's simplicity: no parameters, a read-only annotation, and a clear enumeration of what is reported. It does not describe the exact output format, but for a zero-parameter diagnostic tool this is a minor gap rather than a blocking omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the schema already fully covers the input surface. The description does not need to add parameter-level detail, and the baseline of 4 for a no-parameter tool is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Reports') and identifies the resource ('what this account and server can do'), then enumerates concrete information categories such as configured mailbox, IMAP capabilities, stored flags, new-mail keyword support, and enabled tool groups. This clearly distinguishes it from the sibling tools, which focus on mailboxes, messages, and attachments rather than server/account configuration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: 'Start here when a call fails for reasons that sound like configuration.' This is a clear trigger condition. It does not explicitly state when not to use it or name alternatives, but the sibling tools are sufficiently distinct that no exclusion is necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description explicitly states 'Does not change the read state' and discloses that the body is 'fenced as untrusted content' with a server-side security assessment. This adds meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences place the core action first, then enrich with behavior and options. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the lack of an output schema, the description enumerates what is returned (headers, text body, security assessment, attachments), notes side effects, and covers the optional thread behavior. Nothing essential is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all three parameters at 100%, so the description is not required to document them. It adds slight value by explaining include_thread as 'listing the surrounding conversation' and tying uid to the listing tools, but this is marginal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Fetches one message by UID' and enumerates the returned content (headers, text body, security assessment, attachments). This clearly differentiates it from list_messages and get_attachments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the use case clear: fetch a single message by UID when full content, security assessment, and attachment list are needed. It does not explicitly state when not to use it or name alternatives, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mailboxesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already signals safety, and the description adds useful behavioral detail: it returns message counts, unseen counts, special-use roles, and folder writability. This goes beyond the annotation without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences cover purpose, output contents, and a key usage directive with no filler. The essential information is front-loaded and every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description fully enumerates what the agent can expect in the response—counts, roles, writability, and a reusable path. It also explains how to apply the result in later calls, making it complete for a parameterless listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so there is nothing for the description to explain. The baseline of 4 applies, and the description appropriately focuses on output usage instead of parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every folder in the account with counts, special-use roles, and message-holding capability. This distinguishes it from message-focused siblings like list_messages and get_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs that the returned 'path' should be reused verbatim wherever a mailbox is required, giving practical guidance for when to use this tool. It does not explicitly name alternatives, but the zero-parameter scope and mailbox-listing purpose make the use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_attachmentsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the readOnlyHint annotation by disclosing that small text/images are returned inline, larger payloads are written to the download directory with a path, and executables are refused even if disguised. It also mentions policy constraints on whether fetching is allowed. This gives an agent important behavioral context that annotations alone do not capture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it explains both modes first, then behavior, then the source of part_id, then the safety constraint. Every sentence adds useful information without padding or repetition. It earns its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description adequately covers what the agent will receive: attachment metadata when listing, inline content for small text/images, or a file path for larger files. It also covers policy restrictions, executables, and the required provenance of part_id. Nothing essential appears to be missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all four parameters with 100% coverage, so the baseline is 3. The description adds meaningful value by explaining that part_id must originate from the same tool's listing call and by clarifying the mode behavior tied to auto/inline/file. This is helpful context beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly distinguishes the tool from siblings by specifying two modes: listing the attachments of a message without part_id and retrieving a single attachment with part_id. It names the resource (attachments of a message) and the verbs (list/returns), and its sibling relationships are obvious from context. This is more specific than the title and leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit in-tool usage guidance: omit part_id to list, provide part_id to fetch a specific attachment, and part_id must come from a prior listing call of this same tool. It does not explicitly mention alternative tools such as get_message for reading message content, but the mode-based instructions are clear enough for an agent to select and invoke the tool correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.