Server Quality Checklist

Tool Scores

• list_foldersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds valuable domain context about Proton's distinction between folders and labels and how they appear over IMAP. This helps an agent interpret results beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: purpose, a key distinction, and a behavioral nuance. It is front-loaded with the main action and contains no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two optional params and no output schema, the description plus schema provides adequate context for use. It lacks details about return format but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional parameter-specific meaning beyond what the schema already documents, which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists folders and labels of the Proton account, using a specific verb and resource. It implicitly distinguishes itself from sibling tools like list_messages by targeting mailboxes rather than messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on what the tool does but does not explicitly mention when to use it versus alternatives. It implies use for retrieving mailbox structure but lacks explicit exclusions or sibling tool comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the readOnlyHint annotation by explaining exactly how the message is processed: HTML to text, images/styling dropped, no external content fetched, truncation with a visible note, and untrusted content marking. This gives a complete safety and behavior profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with the core action, followed by important behavioral details. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool lacks an output schema and has 4 parameters, the description covers the essential return behavior (readable text, truncation, untrusted marking) and relevant processing. It could benefit from mentioning what other fields are returned (e.g., subject), but the current coverage is solid for an agent's decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already fully documented. The description adds useful context about long-body truncation that relates to maxChars and textOffset, but it doesn't add meaning to each parameter individually. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Reads one message by its id and returns it as readable text,' specifying the verb, resource, and selection method. This distinguishes it from sibling tools like list_messages and search_messages, which operate on collections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you already have a message id, and the schema reinforces this by noting the id comes from list_messages or search. It doesn't explicitly compare against alternatives, but the context is clear enough for an agent to infer when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses output format (headers only), ordering (newest first), paging, performance characteristics, and scope of 'All Mail'. This significantly extends beyond the readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no redundancy. Front-loaded with core function and each sentence adds essential context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description explains return shape and paging. It covers performance and mailbox scope, making the tool's behavior sufficiently clear for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers all parameters with descriptions (100%). Description adds context that 'text' is slow and groups criteria types, aiding understanding without duplicating schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it searches a mailbox by multiple criteria, using a specific verb and resource. It distinguishes from siblings through criteria and header-only return, but does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States 'At least one criterion is required', implying use for filtered searches. Provides performance guidance and 'All Mail' scope, but does not explicitly contrast with list_messages or other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_attachmentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description discloses significant behaviors: textual attachments return as text, binary ones are refused with a reason, and long attachments can be continued with offset. It also explains that binary content is not dumped into context, which is valuable for context management.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the action and resource. Every clause adds value, with no repetition of schema or annotation details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description adequately covers return behavior (text vs. binary refusal), offset continuation, and the source of the index. It gives enough information for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all 4 parameters (100% coverage). The description adds minimal new parameter information, mainly reinforcing that the index comes from get_message, which the schema already states. Thus a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Reads one attachment of a message by index') and clarifies how the index is obtained from get_message, distinguishing it from sibling tools that list or search messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly explains that the index comes from get_message, implying a prerequisite workflow, and provides a when-not by noting binary attachments are refused. This gives clear usage context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_messagesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds useful behavioral details: stable ordering across pages, headers-only returns, and a latency caveat for large mailboxes. It discloses why bodies are excluded without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is five sentences, each adding value: purpose, ordering, return format, reasoning for headers-only, alternative, and performance caveat. It is front-loaded and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the schema covers parameters, annotations cover safety, and no output schema is needed, the description fully covers return format, ordering, pagination, performance, and alternatives. It is complete for a listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter already well-documented. The description adds context about ordering stability across pages, which relates to limit/offset, but does not significantly redefine parameters. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists mailbox messages newest first, returns headers only, and distinguishes itself from get_message (for bodies) and list_folders. It is specific about the resource and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use get_message with an id to read a message body, and notes this tool never returns bodies. It also mentions performance behavior for large mailboxes, giving clear when-to-use context and an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_configurationA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description discloses that the address contains an access token, changes on server restart, and is valid for the server's lifetime. This is important behavioral context that helps the agent handle the returned value safely and appropriately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every clause contributes meaning. It avoids redundancy and is well-structured for quick parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is complete: it explains what is returned, when to use it, and the important caveats about the token and restart behavior. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter semantics to explain. The description adds value by clarifying the nature of the returned address and its token, which is sufficient given the absence of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('returns') and resource ('address of the local configuration interface'), clearly distinguishing the tool from sibling tools that deal with messages and folders. It also explains the purpose of the interface, leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool: when the user asks how to sign in, where to change settings, or when another tool reports no credentials are available. This provides clear context for invocation and implicitly separates it from the message/folder tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.