Server Quality Checklist

Tool Scores

• page_listA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses pagination order (newest first), offset semantics, and limit cap. With no annotations, the description carries full burden; still lacks return format and edge-case behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sections: purpose sentence and concise arg list with examples. No superfluous text, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Good coverage of purpose and params, but no output schema or return value description. For a simple pagination tool, it is mostly complete, but return format would strengthen it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has zero descriptions (0% coverage), and the description fully explains all three parameters with examples and constraints, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool pages through a list's messages newest first, with specific verb and resource. Distinguishes from siblings like search_list and get_message by focusing on sequential paging.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use for paging through messages but does not explicitly contrast with siblings. No when-not-to-use or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that include_body defaults to True and max_body_chars truncates the body, which is useful behavioral detail. However, it doesn't mention what happens if the UID is not found, or any other side effects or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single clear purpose sentence followed by a bullet-like arg list. Every sentence earns its place, no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description might need to cover return format, but it's a simple get operation. The parameters are fully documented, usage context is clear, and behavioral details like truncation are provided. Minor gap: no mention of error/not-found behavior, but overall it's adequate for a straightforward tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description fully compensates by explaining each parameter: list_name is the mailbox/list name, uid is from search/page results, include_body controls body inclusion, max_body_chars truncates the body. This adds meaningful semantics beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb+resource: "Fetch a single message by UID". It clearly distinguishes this from sibling tools (search_list, page_list) by stating it retrieves one message based on a UID from a prior search/page result.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the UID must come from a prior search/page result, which implies the tool should be used after those. It gives clear context for when to use the tool, though it doesn't explicitly mention when not to use it or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mailing_listsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable context beyond the schema by noting the 'hard-capped at 100' limit for the 'limit' parameter, which is a concrete behavioral constraint. It also clarifies that the mailboxes are 'available over IMAP'. However, it does not mention permissions, error behavior, or whether the operation is read-only, though for a listing tool this is less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence purpose statement followed by a compact Args section. Every line contributes meaningful information without redundancy or fluff. It is appropriately sized for a tool with two simple parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two parameters and no output schema, and the description covers its purpose and parameters well. It mentions the hard-cap for limit, which is important. However, it lacks any mention of the return format or pagination behavior, which could be relevant for a listing operation. Given the tool's simplicity, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% coverage, but the description fully compensates by explaining both parameters. For 'pattern', it provides IMAP LIST patterns with examples like '"agent*"' and '"*"'. For 'limit', it clarifies meaning and 'hard-capped at 100'. This adds semantic depth beyond the schema's type/default fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List IETF mailing-list mailboxes available over IMAP.' This provides a specific verb ('List'), a specific resource ('IETF mailing-list mailboxes'), and a specific context ('available over IMAP'), which distinguishes it from sibling tools like search_list or get_message that operate on messages rather than mailboxes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear purpose but does not explicitly state when to use this tool over alternatives. It does not mention any conditions, prerequisites, or exclusions. The usage context is implied through the tool name and description, but there is no direct comparison to sibling tools, so the guidance is limited.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses important behaviors: results are summaries, sorted newest first, limit defaults to 25 and hard-capped at 100. It doesn't mention pagination or error handling, but the described behavior is sufficient for basic usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact, well-structured docstring with a clear summary line followed by a parameter list. Every sentence is informative, no fluff, and the key behavior (newest first, default limit, cap) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must cover return behavior. It states 'matching message summaries' and ordering, which is mostly adequate, though it could mention whether summaries contain message IDs or how to fetch full messages. Still, given the presence of sibling get_message, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully explain parameters. It does so excellently: each parameter has a clear semantic ('substring to match', 'free-text across whole message', 'ISO date'), plus helpfully documents defaults and limits that aren't in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a mailing list and returns matching message summaries, with an explicit ordering (newest first). This is a specific verb+resource combination that distinguishes it from sibling tools like list_mailing_lists, page_list, and get_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it's for searching across messages in a mailing list with various filters. It doesn't explicitly state when to prefer this over alternatives, but the purpose is unambiguous from the name and description, giving the agent enough guidance to choose it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.