Server Quality Checklist

Tool Scores

• list_mailboxesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden for behavioral disclosure. It discloses the output format (lines of '<Konto>:<Mailbox>'), which is useful, and implies it's a read-only listing operation. However, it doesn't disclose ordering, pagination, or whether mailboxes are hierarchical, leaving some behavioral gaps for what should be a simple listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action. Every sentence adds value: the first states the action and output format, the second explains the purpose (finding exact names for use in other tools). Minor opportunity to be even more compact, but it's efficient with no wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only listing tool with no output schema, the description covers the essential aspects: what it lists, output format, and why to use it. The absence of return-type documentation is acceptable since the description itself defines the output shape. It's reasonably complete for a simple discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema coverage, the schema carries no semantic burden and the description is not required to add parameter meaning. The description appropriately explains what the tool returns (names in '<Konto>:<Mailbox>' format), which compensates for the absent parameters by explaining the output structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists all Apple Mail accounts and mailboxes as '<Konto>:<Mailbox>' lines, explaining the exact output format. It distinguishes its purpose as discovery of exact names. However, it doesn't explicitly differentiate from siblings like search_messages or get_message_body, though the output-format detail is strong.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the user this tool is used to find exact names for the 'mailboxes' parameter of other tools, which establishes a clear usage context. It doesn't list explicit exclusions or alternatives, but the sibling tools are clearly distinct operations (saving, fetching bodies, flagging, searching), so the usage intent is well communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_attachmentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does disclose that it creates the destination folder ('Der Zielordner wird angelegt') and that it saves only the first matching attachment, plus the sorting/exclusion behavior. However, it doesn't disclose write semantics, required permissions, or side effects beyond folder creation. It's a mutation tool with no annotation coverage, so a 3 is appropriate—decent but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, all substantive. First sentence states the core action, second explains matching and exclusions, third states folder creation. No filler, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 100% schema coverage, no output schema, and no annotations, the description is reasonably complete. It explains matching, exclusions, and folder creation. It could add a return-value note or failure behavior (e.g., what happens if no attachment matches), but given the constraints this is solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema documents all four parameters. The description adds matching semantics for attKey (contains, .pdf suffix) and subjKey (Betreff match). The mailboxes parameter says 'Vorgabe wie bei search_messages' deferring to sibling. This adds some value beyond schema but leans on the schema's full coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb+resource ('Speichert den ersten passenden PDF-Anhang') and clearly specifies the object target ('einer über den Betreff gefundenen Nachricht'). It distinguishes from siblings by its unique purpose (attachment saving vs. messaging/search operations), and even names the sibling search_messages. Very clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the matching criteria precisely: attachment must contain attKey and end in .pdf, explicitly excluding AGB, promotional PDFs, XML, and images. It names the alternative search_messages for the mailbox lookup default. However, it doesn't explicitly state when NOT to use this tool relative to get_message_body or flag_message (though the distinction is somewhat obvious).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_message_bodyA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it returns 'der ersten passenden Nachricht' (first matching message only), which is a meaningful selectivity behavior, and mentions the Kopf fields returned. It does not disclose error behavior when no message matches, or pagination/truncation limits, but the core selectivity disclosure is valuable given zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact three-sentence German paragraph that efficiently covers purpose, use case, and a required-parameter constraint. It's front-loaded with the core function. Slightly more text than strictly necessary but no wasted sentences or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description helpfully lists the return content (From, Subject, Date header fields plus Klartext-Inhalt). The tool has moderate complexity with 4 optional parameters and no required ones; the description covers the key conditional requirement and use-case, though it could note what happens when no message matches or when both keys are absent. The output-format disclosure compensates well for the missing output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters. The description adds the constraint that at least one of subjKey or senderKey must be provided ('Mindestens subjKey oder senderKey angeben'), which is meaningful semantic value beyond the schema, and explains the mailboxes default references search_messages. Since coverage is complete, baseline 3 is appropriate with the added conditional requirement noted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Liefert den Klartext-Inhalt'), the specific resource ('der ersten passenden Nachricht'), and includes the returned content fields (Kopf with From, Subject, Date). It distinguishes its purpose from siblings by explicitly targeting messages where evidence lives only in the mail body without a PDF attachment, differentiating it from save_attachment and search_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use it ('Für Belege, die nur im Mailtext stehen und kein PDF anhängen'), giving clear selection context relative to alternatives. However, it lacks explicit when-NOT-to-use statements or named alternative tools as exclusions, and does not mention how mailboxes default differs from search_messages beyond a passing note in the schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flag_messageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and it delivers: it discloses mutation ('Verändert die Nachricht'), the ambiguity policy (multi-match → no change, report ambiguity), the flagIndex constraint (only applicable when flagged=true), and the color semantics. It does not mention auth requirements or reversibility beyond the unflag toggle, which keeps it slightly below 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured single block that front-loads the core action, then covers edge cases and use case. Every sentence earns its place—no filler. Slightly verbose with the parenthetical color mapping and schema redundancy, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no output schema and no annotations, the description covers the key behaviors: mutation, ambiguity handling, minimal parameter requirements, and a concrete use case. It could mention prerequisites (existing message) or what the return value signifies (success vs ambiguity), but the guidance is solid for selecting and invoking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all 5 parameters are documented in the schema. The description adds minimal extra value: it maps flagIndex to actual color names and states the minimum requirement (subjKey or senderKey), which slightly enriches the schema. Baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb+resource: 'Setzt oder entfernt die Fahne an GENAU EINER Nachricht' clearly states it flags/unflags exactly one message. It discloses the flag color range (0-6: rot, orange, gelb, grün, blau, lila, grau) and distinguishes itself from search_messages/siblings by focusing on mutation of a single matched message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the practical use case ('Praktisch, um eine verarbeitete Rechnung zu markieren'), describes the when-not behavior (if key matches multiple messages, nothing is changed and ambiguity is reported), and provides parameter guidance (mindestens subjKey oder senderKey angeben). This clearly differentiates from read-only siblings like search_messages and get_message_body.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that results are deduplicated via message id, that attachment names appear as ATT{...}, and flags as ⚑. It also reveals the behavioral note about returning everything if no filters are given — a genuine edge-case behavior worth disclosing. This is solid behavioral context for a read-only search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient — roughly three sentences covering purpose, output format markers, filter behavior, and deduplication. It front-loads the core purpose. Each sentence carries meaningful content with no wasted words. Slightly longer than the ideal two-sentence description but earns its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool with 100% schema coverage and no output schema, the description covers the essential aspects: what's searched, what's returned (with marker notation), filter semantics, and deduplication. The only minor gap is that it doesn't describe result ordering or pagination/limits, but for a list-returning search tool this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and every parameter has a description in the schema. The description adds semantics that complement the schema: it explains ATT{...} output naming and deduplication, and clarifies the filter interplay (that vendors and flaggedOnly are the two filter paths). The description doesn't need to re-explain each parameter since the schema already does so thoroughly at 100% coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches messages in a time window, lists hits with attachment names (ATT{...}) and flag markers (⚑). It explicitly distinguishes what it does — filtering by sender keywords and/or flagged messages, deduplicating by message id. This is a specific verb+resource with clear scope that differentiates from the sibling tools (save_attachment, get_message_body, flag_message, list_mailboxes), which all perform clearly different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: it explains that it filters by sender keywords and/or flagged-only messages, and explicitly states 'ohne einen der beiden Filter käme alles zurück' (without either filter, everything would be returned). This helps the agent know when filtering is appropriate. However, it doesn't name explicit alternatives or state when NOT to use this tool in favor of a sibling, though the sibling differentiation is reasonably clear from names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.