Server Quality Checklist

Tool Scores

• create_draftC
  2.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The 'Behavioral Transparency' section claims the tool is read-only with no side effects, but saving a draft inherently modifies state. This contradicts the tool's actual behavior. The detailed security/rate-limit information is copied from a different type of tool and does not fit an email draft creator.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy and contains irrelevant sections (e.g., behavioral transparency for a different tool). Information is repeated and not front-loaded. The first paragraph is concise, but the rest is unnecessary and misleading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no output schema, and no annotations, the description fails to provide complete guidance. It does not explain how to use the tool, what the response looks like, or what happens after saving. The contradictory sections make it unusable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists 'to', 'subject', 'body' in the Args section but provides no format, constraints, or examples. The 'api_key' parameter is not mentioned. With 0% schema description coverage, the description adds minimal value beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states 'Save an email as a draft without sending it', specifying the verb (save), resource (draft), and distinct action (not sending). This distinguishes it from 'send_email' among siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section is generic and unrelated ('structured analysis or classification'), and the 'When NOT to use' mentions 'real-time production decision-making', which does not apply to drafting emails. These sections are misleading and contradict the tool's purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_emailC
  2.4/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. It claims the tool is 'read-only and stateless' and 'does not modify any external systems', which is false for an email-sending tool. This is a critical misrepresentation. Other details (rate limits, auth, error handling) are overshadowed by this contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly long and includes irrelevant sections (classification use cases) that likely were copy-pasted. The relevant email-sending info is mixed with off-topic content. Not concise; structure is poor.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers many aspects (safety, rate limits, auth, error handling) but the false read-only claim undermines credibility. Missing output schema. The confusing unrelated sections reduce completeness for the actual task.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It lists all 8 parameters with clear explanations: to/subject/body as plain text, html boolean, cc/bcc as comma-separated, confirm for preview vs send. The api_key parameter is mentioned in behavioral sections. Adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states 'Send an email via SMTP.' However, subsequent contradictory statements (e.g., 'read-only and stateless') and irrelevant 'When to use' sections about classification frameworks confuse the core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes explicit 'When to use' and 'When NOT to use' sections, but these are copied from an unrelated tool (classification analysis). No comparison to siblings like create_draft or search_emails. The safety note about confirm parameter is helpful but insufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_foldersB
  3.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It covers read-only, side effects, authentication, rate limits, error handling, idempotency, and data privacy. However, inconsistencies with the generic text reduce clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Overly verbose with repetitive and irrelevant sections (e.g., generic behavioral text copied from another tool). Could be significantly shorter without losing essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers many behavioral aspects (rate limits, error handling, etc.) but the irrelevant sections undermine completeness. For a simple list tool, the key purpose and behavior are adequately described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (api_key) with schema coverage 0%. The description explains it's optional for basic usage and required for higher tiers, adding meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool lists mailbox folders (INBOX, Sent, etc.), but later generic text about 'structured analysis or classification' contradicts the actual purpose and confuses the agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides some context ('Useful for discovering folder names before reading or searching'), but includes contradictory guidance recommending the tool for 'structured analysis' which is misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_emailsA
  3.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly states read-only, stateless, idempotent behavior, authentication requirements, rate limits (free vs. pro), error handling, and data privacy. This provides comprehensive behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with sections but is verbose and contains redundant information (Behavior and Behavioral Transparency sections overlap). The generic 'When to use' section could be removed. Concision is adequate but not optimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description should explain return structure. It covers error objects but not success output. It also doesn't mention how api_key is used for authentication. These gaps reduce completeness despite good coverage of query formats and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It details query, folder (with default), and limit (with default and max), but misses the api_key parameter entirely. Query format explanation is thorough, but omission of api_key reduces score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search emails in a folder' with specific and varied query formats, distinguishing it from sibling tools like create_draft, list_folders, read_inbox, and send_email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section is generic and unrelated to searching emails, and there is no guidance on when to use this tool versus alternatives like read_inbox or list_folders. The query format examples offer implicit usage context but no explicit differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_inboxA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description provides a comprehensive 'Behavioral Transparency' section covering side effects (read-only, no side effects), authentication, rate limits, error handling, idempotency, and data privacy. This fully informs the agent of behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy with structured sections, but some parts (e.g., 'When to use' reads like a generic template) add redundancy. While well-organized, it could be more concise by removing generic statements and focusing on email-specific details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains return format and behavior. However, it omits details about pagination or handling large folders, and the missing api_key parameter explanation reduces completeness. Error handling and rate limits are covered well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. The 'Args' section explains folder and limit (defaults and max), but does not mention the api_key parameter present in the schema. This omission leaves ambiguity about how authentication is handled via parameter vs environment variable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that it reads recent emails from a mailbox folder and returns subject, from, date, and body preview. This distinguishes it from sibling tools like create_draft, list_folders, search_emails, and send_email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'When to use' and 'When NOT to use' sections, but the content is generic ('structured analysis or classification') and not specific to reading emails. It lacks explicit differentiation from siblings or clear guidance on when to use this tool versus search_emails or others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.