Server Quality Checklist

Tool Scores

• save_gmail_draftC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only states it never sends and returns draft id. Missing details: overwrite behavior, permissions, rate limits, thread_id usage for replies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with key action and side effect. Could be slightly expanded to cover parameter roles without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 4 parameters, optional thread_id, and an output schema. Description omits crucial context: reply vs new draft, thread_id function, limits, and response details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no parameter descriptions). The description does not explain any parameter beyond their names. 'thread_id' is especially ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Verbs 'Save' and 'never sends' clearly define the action. Resource 'Gmail DRAFT' is specific. Differentiates from siblings like fetch_emails (read) and write_report (different output).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. No mention of prerequisites like authentication or Gmail account. No comparison to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_reportB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only says 'write to disk' without details on overwrites, permissions, error handling, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write tool, the description covers the core action but omits details like file overwrite behavior. An output schema exists, so return value is not needed. Still, behavioral gaps reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds no meaning beyond the schema. It does not explain what the markdown string should contain or how the path defaults work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'write' and the resource 'final triage report markdown', with target 'disk'. It distinguishes from siblings like fetch_emails and append_tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives, but the task is well-defined and siblings are unrelated. The 'final' qualifier implies use at report completion, but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• append_tasksA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states 'Append tasks' – it does not disclose whether the file is created if missing, how existing content is handled, or what the return value is. Behavioral traits like side effects or error conditions are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two short sentences, with no wasted words. Every part is necessary and front-loaded: first sentence states action, second sentence specifies format. Ideal for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple operation (append tasks to a file) and the availability of an output schema, the description is minimally adequate. However, it omits details like file creation behavior and error handling, which could be important for agent decision-making. It covers the essential purpose but lacks completeness for a robust tool experience.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and defines items as an array of objects with 'additionalProperties: true', which is very permissive. The description provides critical structure by specifying 'Each item: {text, deadline?}' – this adds meaningful constraints beyond the schema and clarifies the intended format. However, the 'path' parameter is not explained beyond the schema default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Append', the target resource 'tasks.md', and the format of each item ('{text, deadline?}'). It is specific and leaves no ambiguity about what the tool does. The sibling tools are unrelated, so no confusion arises.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. Sibling tools are for different purposes, but the description does not include any contextual cues or prerequisites, leaving the agent to infer usage from naming alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_emailsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output format ('JSON array of {id, sender, ...}') and implies a read-only operation ('fetch'). It also instructs the agent to process the emails, adding behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences and reasonably concise, but contains extraneous workflow instructions ('Reason over these yourself...') that could be omitted or placed elsewhere. It is front-loaded but includes instructions better suited for a prompt.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 optional parameters) and the presence of an output schema, the description captures the main purpose and return format. However, missing explanation for the 'since' parameter and lack of error handling details reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description only mentions the 'mock' parameter ('Use mock=True for the fixture demo'), but does not explain the 'since' parameter, which has a default of '24h'. The agent is left to infer its purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch unread emails (JSON list)', using a specific verb and resource. This distinguishes it from sibling tools like append_tasks, save_gmail_draft, and write_report, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a concrete usage scenario ('Use mock=True for the fixture demo') and implies a workflow ('then call the IO tools'). However, it does not explicitly state when not to use this tool or compare it to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.