Server Quality Checklist

Tool Scores

• create_draftA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (all false), the description discloses the non-sending behavior and the result echoing the recipient list, which are crucial for an email tool. It does not mention permissions or rate limits, but the provided details are valuable and do not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no redundancy, front-loaded with the primary action and immediately followed by the most important behavioral nuance (never sent). Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description partially explains the return behavior (echoes recipient list) and the post-save state. It does not cover attachment handling or errors, but given the tool's moderate complexity, the provided context is sufficient for an agent to use it safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20% (attachments have a description), and the tool description adds minimal parameter meaning beyond referencing the 'recipient list.' It does not explain subject/body semantics or required parameters, leaving the burden on a sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Save a draft') and resource ('Drafts folder'), clearly differentiating this write tool from the read-only sibling tools. The verb and object are unambiguous, and the 'never sent' detail further distinguishes its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (creating a draft) and emphasizes it does not send emails, but it does not explicitly mention alternatives or when not to use it. The context of sibling read tools makes the distinction clear, but it stops short of explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_attachmentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no useful annotation hints (all false), the description carries the burden of disclosing side effects. It clearly states that the file is written to disk and that its content is not read into the conversation, which is important behavioral information. It could mention overwrite behavior or error handling, but for a simple tool it is quite transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear sentences, no filler. The first sentence states the action and outcome, the second clarifies the side effect. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no output schema and weak annotations, the description covers the essential behavior: downloading to a directory and providing the file location. It lacks details about error conditions or prerequisites, but given the tool's simplicity, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is only 33% (only part_id has inline description), yet the description does not explain the other two parameters (folder, uid). It adds no extra meaning about what these parameters represent or how they relate to the download process, leaving the agent to guess their purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'download' and identifies the resource as 'one attachment' with a clear destination ('configured download directory'). It also states the outcome ('report where it landed'), which distinguishes it from siblings like get_email or search_emails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool (when you need an attachment saved to disk) and even clarifies that the bytes are not read into the conversation, which implies it is not for content inspection. However, it does not explicitly name alternative tools or state exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_emailA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses several non-obvious behaviors: 64 kB truncation, HTML-to-text conversion, attachment listing without download, and a Warnings line for suspicious content. It also includes an important security caveat about treating message text as data. These go well beyond the readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences that are dense with useful information. The opening 'Read one message' is a clear, front-loaded purpose, and every subsequent clause adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite minimal schema and no output schema, the description gives a comprehensive picture of the tool's behavior: input format (one message), output characteristics (truncated text, warnings), and a security note. It is sufficient for an agent to use the tool safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain the 'folder' and 'uid' parameters or how they identify the message. With 0% schema coverage, the description was responsible for compensating, but it remains silent on parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Read' with the resource 'one message' and details the content (envelope, body). It clearly distinguishes from siblings like get_attachment (attachments not downloaded), search_emails (search), and create_draft (write).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for reading a single email's content and explicitly notes attachments are listed but not downloaded, subtly directing users to get_attachment for actual attachment retrieval. However, it lacks explicit when-to-use/when-not-to-use statements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_emailsA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description discloses the sort order ('newest first'), the limitation to metadata only, and a security-relevant behavior about message content being potential instructions. This adds significant context that the annotations alone do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is remarkably concise: three sentences, each earning its place. The first states the core purpose, the second adds key behavioral details, and the third provides a vital security guideline. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 10 parameters and no output schema, the description is somewhat incomplete. It mentions 'metadata only' but does not specify which metadata fields are returned, nor does it mention pagination behavior beyond what the schema defaults imply. The security note is a plus, but the lack of return-value detail is a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only 40% parameter description coverage, and the tool description does not compensate. It does not explain the meaning or usage of parameters like to, from, limit, offset, subject, or unread_only, leaving the agent without adequate guidance for these fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Search') and resource ('the mailbox'). It distinguishes itself from siblings by explicitly noting it returns 'metadata only (no message bodies)', which sets it apart from get_email that presumably retrieves full content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (searching the mailbox) and includes a crucial usage guideline about treating message content as untrusted data. However, it does not explicitly mention alternatives like get_email for retrieving full bodies, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_foldersA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is known. The description adds behavioral detail by specifying that the output includes message counts per folder, going beyond what the tool name implies. It does not discuss nesting or ordering, but this is sufficient for a simple list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the verb and resource. Every word adds value, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter, read-only nature of the tool and the absence of an output schema, the description is complete enough. It clearly states what the tool returns (folders with message counts), and the annotations cover side effects and world assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already covers 100% of parameters. Per the baseline for 0 parameters, a score of 4 is appropriate; no parameter description is needed since there are no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with the resource 'mailbox folders' and adds the distinguishing detail 'with their message counts', making it clear what the tool does. This clearly differentiates it from sibling tools like search_emails or get_email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided about when to use this tool versus alternatives. The usage is implied from the name and description (to get an overview of folders and their counts), but there are no exclusions or references to sibling tools, so it stops at implied usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.