Server Quality Checklist

Tool Scores

• messagesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover basic hints (readOnlyHint: false, destructiveHint: false, etc.), so the description adds value by detailing specific actions and query syntax. However, it doesn't disclose additional behavioral traits like rate limits, authentication needs, or error handling beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with key actions, though it could be more structured (e.g., separating action types). Every sentence adds value, but minor improvements in organization could enhance clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of 8 parameters, no output schema, and annotations covering basic hints, the description is adequate but incomplete. It lacks details on return values, error cases, or prerequisites, leaving gaps for an AI agent to fully understand tool behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters. The description adds minimal semantics by listing actions and query examples, but doesn't provide significant meaning beyond what's in the schema, aligning with the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool handles email messages with specific actions (search, read, mark_read, etc.), which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'attachments' or 'folders', though the domain focus is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through action enumeration and query examples, but doesn't provide explicit guidance on when to use this tool versus alternatives like 'send' or 'attachments'. It mentions filtering by account, which offers some context, but lacks clear when/when-not directives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• attachmentsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, and idempotent behavior, which the description does not contradict. The description adds valuable context beyond annotations by specifying that download returns 'base64-encoded content,' which is a key behavioral trait not covered by annotations. However, it does not mention rate limits or auth needs, which could be relevant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of two efficient sentences that directly state the tool's actions and outcomes without any wasted words. Each sentence earns its place by clarifying the tool's functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no output schema) and rich annotations, the description is mostly complete. It covers the core actions and output format for download, but could benefit from mentioning error cases or response structures for list actions. However, annotations provide safety context, making it adequate overall.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal semantic value beyond the schema by mentioning 'list shows all attachments for an email' and 'download returns base64-encoded content,' which loosely relates to parameters like 'action' and 'filename,' but does not provide additional details on parameter usage or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('list, download') and resource ('email attachments'), and distinguishes it from sibling tools like 'messages' or 'send' by focusing exclusively on attachment operations. It explicitly mentions what each action does: 'List shows all attachments for an email' and 'Download returns base64-encoded content.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying actions ('list' and 'download') and their outcomes, but does not explicitly state when to use this tool versus alternatives like 'messages' for general email handling. It provides some context (e.g., 'list shows all attachments for an email'), but lacks explicit guidance on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sendA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide key behavioral hints (readOnlyHint=false, destructiveHint=false, etc.), and the description adds useful context beyond this, such as how reply and forward actions handle subject lines and thread headers. However, it lacks details on potential side effects, error conditions, or rate limits, which would enhance transparency further.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured, using two sentences that efficiently cover the tool's actions and key behavioral details without redundancy. Every sentence adds value, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, no output schema) and rich annotations, the description is mostly complete, covering core functionality and behavioral traits. However, it could be more comprehensive by including information on error handling or response formats, which would improve completeness for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents all parameters thoroughly. The description adds minimal semantic value by mentioning 'reply maintains thread headers' and 'forward includes original body', but does not elaborate on parameter interactions or usage nuances beyond what the schema provides, aligning with the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('send emails: new, reply, forward') and distinguishes it from sibling tools like 'attachments', 'folders', 'help', and 'messages' by focusing exclusively on email sending functionality. It provides concrete details about how reply and forward actions work, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides usage by explaining the differences between 'new', 'reply', and 'forward' actions, including how reply maintains thread headers and auto-prepends 'Re:', and forward includes original body and auto-prepends 'Fwd:'. However, it does not explicitly state when to use this tool versus alternatives or mention any exclusions, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• foldersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, and idempotent behavior, which the description doesn't contradict. The description adds value by specifying the return data ('folder names, paths, and flags'), which isn't covered by annotations, providing useful context about what information is retrieved. No rate limits or auth needs are mentioned, but annotations cover safety aspects adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently states the action, scope, and output without unnecessary words. It is front-loaded with the core purpose and avoids redundancy, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (list operation), rich annotations (read-only, idempotent), and full schema coverage, the description is mostly complete. It specifies output details, which compensates for the lack of an output schema. However, it could improve by mentioning any limitations (e.g., pagination) or error cases, though annotations provide good safety context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (action and account). The description mentions 'one or all email accounts,' which aligns with the optional account parameter but doesn't add syntax or format details beyond what the schema provides. With high schema coverage, baseline 3 is appropriate as the description adds minimal extra semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('mailbox folders'), specifies scope ('for one or all email accounts'), and distinguishes from siblings like 'messages' or 'attachments' by focusing on folder metadata rather than message content. It provides specific output details ('folder names, paths, and flags') that further clarify its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it works for 'one or all email accounts,' which helps differentiate from tools that might target specific accounts only. However, it lacks explicit guidance on when to use this versus alternatives like 'messages' (which might list messages within folders) or any prerequisites for accessing accounts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• helpA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable context beyond what annotations provide. While annotations indicate readOnlyHint=true, idempotentHint=true, and destructiveHint=false, the description clarifies that this tool provides 'full documentation' when regular descriptions are insufficient. This adds meaningful behavioral context about the tool's purpose and output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each serve distinct purposes: the first states what the tool does, the second provides usage guidance. There's zero wasted language and it's front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple purpose (retrieving documentation), single parameter with full schema coverage, and comprehensive annotations, the description is mostly complete. However, it doesn't describe what format the 'full documentation' returns (e.g., markdown, HTML, plain text), which would be helpful since there's no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already fully documents the single parameter 'tool_name' with its enum values. The description doesn't add any additional parameter semantics beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get full documentation') and resource ('for a tool'), distinguishing it from sibling tools like 'messages', 'folders', 'attachments', and 'send' which perform different operations. It explicitly defines its unique purpose in the tool ecosystem.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Use when compressed descriptions are insufficient.' This clearly differentiates it from the default tool descriptions and provides a specific trigger condition for its usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.