Server Quality Checklist

Tool Scores

• open_archiveA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full transparency burden. It discloses key behaviors: 'entirely locally, nothing leaves the machine' (privacy), returns 'message count, date range, and top senders', and 'Large archives are indexed once and cached'. It doesn't detail side effects like where the index/cache is stored, but provides meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, each adding distinct value: input formats + privacy, return values, and caching behavior. It is front-loaded with the core purpose and contains no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one required parameter and no output schema, the description is remarkably complete. It covers input types, local processing guarantee, summary statistics returned, and caching behavior. No critical gaps for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the 'path' parameter is fully described in the schema. The description restates similar information ('.mbox file' and '.eml' directory) but adds no new parameter semantics beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Index' with a clear resource: '.mbox file' or 'directory of .eml files'. It distinguishes itself from sibling tools (search_messages, get_message) by focusing on loading/opening an entire archive rather than retrieving individual messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: to index an email archive for local processing. It provides clear context (supports Google Takeout and Thunderbird exports) but does not explicitly contrast with siblings or mention when not to use it. No exclusions or alternative tool names are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It clearly states that the body is decoded, may be truncated if huge, and that attachment contents are never returned. These are non-obvious behaviors that could surprise users, making this disclosure valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and scope. Every clause adds value: what is parsed, what is truncated, and what is excluded. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema and no annotations, the description provides a complete contract. It specifies what is returned (headers, decoded body, attachment metadata), the truncation behavior, and the explicit exclusion of attachment contents. This is sufficient for an agent to invoke the tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters, with 'id' and 'path' already described in the input schema. The description adds no new parameter-level meaning beyond referencing 'from search_messages', which is already implied by the schema. Since the schema fully covers parameter semantics, the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fully parse') and identifies the exact resource ('one message by its id'). It also distinguishes itself from sibling tools by referencing search_messages as the source of the id, making it clear this tool retrieves individual message details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a workflow: get an id from search_messages, then use get_message for full parsing. It also sets a clear limitation ('Attachment contents are never returned'), which suggests not using this tool for attachment content. However, it does not explicitly name alternatives or state when not to use beyond the attachment caveat.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that query matches sender and subject, that body_query full-parses candidates, that it is capped, and that the result reports scanned candidates. This goes beyond basic read/write safety and explains performance-relevant behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, followed by a focused explanation of the key parameter behavior. Every sentence earns its place without repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters and no output schema, and the description covers the essential behavioral aspects: what is indexed, how query works, how body_query differs, and the cap/results. It does not fully describe the result structure beyond the scanned count, which would have been helpful without an output schema, but it is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds practical meaning to body_query by explaining its capped, full-parse nature and the advice to narrow filters first, which the schema does not convey. It also groups query as matching sender and subject, reinforcing schema but adding little new beyond body_query.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'indexed envelopes' with a scope of from/to/subject/date. It differentiates from siblings like open_archive and get_message by focusing on searching across messages, not opening or retrieving a single message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance for the body_query parameter, advising to narrow with sender/date filters first due to the cap. It does not explicitly state when to use this tool vs alternatives, but the purpose is clear enough that an agent would know to use it for searching rather than opening or retrieving.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.