Server Quality Checklist

Tool Scores

• create_reply_draftC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates a draft, implying a write operation, but doesn't disclose key traits like required permissions, whether the draft is saved automatically, rate limits, or error handling. It mentions a return status but lacks details on success/failure conditions. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first. The 'Args' and 'Returns' sections add structure, though they could be integrated more seamlessly. There's no wasted text, but minor improvements in flow could enhance readability, keeping it efficient but not perfect.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, mutation operation), no annotations, and an output schema present, the description is partially complete. It covers the basic action and parameters but lacks behavioral context and detailed usage guidelines. The output schema likely handles return values, so the description's mention of 'A status message' is sufficient, but overall gaps remain in transparency and guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantics beyond the input schema. It explains that 'email_id' is 'The ID of the email to reply to' and 'reply_text' is 'The text content of the reply,' which clarifies purpose but doesn't provide format details, constraints, or examples. With 0% schema description coverage, the description partially compensates but not fully, aligning with the baseline of 3 when schema coverage is low but some param info is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a reply draft for an email with the given reply text.' This specifies the verb ('create'), resource ('reply draft'), and target ('email'). It distinguishes from sibling 'create_new_email_draft' (new email vs reply) and 'read_emails' (read vs create), though not explicitly. It's not a tautology but lacks explicit sibling differentiation, so it's a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to use 'create_reply_draft' instead of 'create_new_email_draft' or other potential tools, nor does it specify prerequisites or exclusions. The context is implied (replying to an email), but no explicit usage rules are given, resulting in a score of 2.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_emailsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool fetches 'recent emails' and handles pagination via 'page_token', which adds useful behavioral context. However, it lacks details on permissions, rate limits, or error handling, which are important for a tool interacting with an external service like Gmail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise, with a clear purpose statement followed by 'Args:' and 'Returns:' sections. Each sentence adds value without redundancy, making it easy to scan and understand the tool's functionality and outputs efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but with an output schema), the description is fairly complete. It explains the purpose, parameters, and return values. The output schema is mentioned, so detailed return value explanations aren't needed. However, it could improve by addressing authentication or error scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics for both parameters: 'max_results' is explained as 'Number of emails to fetch,' and 'page_token' as 'Optional page token to fetch next batch.' This compensates for the 0% schema description coverage by clarifying the purpose and usage of each parameter beyond their types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetch recent emails from Gmail inbox.' It specifies the verb ('fetch') and resource ('emails from Gmail inbox'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'create_new_email_draft' or 'create_reply_draft', which are write operations, whereas this is a read operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the context of fetching emails, but it doesn't explicitly state when to use this tool versus alternatives. There are no guidelines on prerequisites, such as authentication or access rights, or exclusions. The presence of sibling tools suggests potential alternatives, but no direct comparison is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_new_email_draftA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions the tool creates a draft (implying a write operation), it doesn't disclose any behavioral traits such as authentication requirements, rate limits, error conditions, or what happens if the draft already exists. The description is minimal beyond stating the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, Args, Returns), uses minimal sentences that each serve a purpose, and is front-loaded with the main functionality. There's no wasted verbiage or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there's an output schema (which handles return values), no annotations, and the description covers all parameters, it's moderately complete. However, for a write operation tool with no annotations, it lacks important behavioral context like permissions needed, side effects, or error handling, which reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by clearly explaining all three parameters (recipient, subject, body_text) in the Args section, adding meaningful context like 'can be empty to leave blank' for recipient and default values implied by 'No Subject' and empty body. This provides good semantic understanding beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new draft email') and distinguishes it from sibling tools by explicitly noting '(not a reply)', differentiating it from 'create_reply_draft'. The verb+resource combination is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating 'not a reply', which helps differentiate when to use this tool versus 'create_reply_draft'. However, it doesn't explicitly mention when to use this versus 'read_emails' or provide any exclusions or prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.