Server Quality Checklist

Tool Scores

• send_emailD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers none. It doesn't mention whether this requires authentication, has rate limits, what happens on failure, whether emails are queued or sent immediately, or any other behavioral characteristics. For a mutation tool with zero annotation coverage, this is a critical gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While extremely concise with just two characters, this represents under-specification rather than effective conciseness. The description doesn't front-load important information and fails to provide the necessary context for a tool with 7 parameters and multiple sibling alternatives.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 7 parameters, no annotations, no output schema, and multiple sibling tools, the description is completely inadequate. It provides no behavioral context, no usage guidance, and minimal purpose clarification beyond the tool name itself.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly with Chinese descriptions. The description adds no additional parameter information beyond what's in the schema, which meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '发送邮件' (Send email) is a tautology that restates the tool name in Chinese. While it clearly indicates the general action, it lacks specificity about what resources or capabilities are involved and doesn't differentiate from sibling tools like 'configure_email_server' or 'get_recent_emails'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are multiple sibling tools for email operations (configure_email_server, get_email_content, get_recent_emails, etc.), but the description offers no context about when this specific send_email tool is appropriate versus those other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• configure_email_serverC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It hints at advanced usage but doesn't specify if this is a write operation (likely, given 'configure'), what permissions are required, whether changes are reversible, or any rate limits. For a configuration tool with 8 parameters and no annotations, this is a significant gap in transparency about its effects and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese: '手动配置邮箱服务器设置（高级用户使用）'. It's front-loaded with the core action and resource, and the parenthetical adds context without verbosity. However, it could be more structured by explicitly separating purpose from guidelines, but it earns a 4 for being appropriately sized with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, configuration tool), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what happens after configuration (e.g., success/failure states, return values), behavioral risks, or integration with sibling tools. For a tool that likely mutates system settings, more context is needed to guide safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all 8 parameters clearly documented in the input schema (e.g., SMTP/IMAP settings, user credentials). The description adds no additional parameter information beyond the schema, such as format examples or interdependencies. Given the high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting without description enhancement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as 'manually configure email server settings (for advanced users)', which is clear but vague. It specifies the action ('configure') and resource ('email server settings'), but lacks detail on what exactly is configured. It distinguishes from siblings like 'send_email' or 'get_email_content' by focusing on configuration, but doesn't explicitly differentiate from 'setup_email_account' or 'test_email_connection', which might overlap in purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance: it mentions 'for advanced users', implying complexity or risk, but offers no explicit when-to-use rules, prerequisites, or alternatives. It doesn't clarify when to use this versus 'setup_email_account' (which might be simpler) or 'test_email_connection' (which might be for validation), leaving the agent with little direction on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_email_contentC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. While '获取' (get) implies a read operation, the description doesn't specify whether this requires authentication, what format the content returns (HTML, plain text, attachments), whether there are rate limits, or what happens with invalid UIDs. For a tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient Chinese sentence that directly states the tool's function. There's zero wasted text - every character contributes to understanding what the tool does. It's appropriately sized for a simple retrieval tool with one parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. While the purpose is clear, there's no information about return format (headers, body, attachments), error conditions, authentication requirements, or how this differs from sibling email tools. The agent would need to guess about important behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the single parameter 'uid' documented as '邮件唯一标识符' (email unique identifier). The description adds no additional parameter information beyond what the schema provides. With high schema coverage and only one parameter, the baseline score of 3 is appropriate - the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '获取指定邮件的详细内容' (Get detailed content of specified email) clearly states the tool's purpose with a specific verb ('获取' - get) and resource ('邮件详细内容' - email detailed content). However, it doesn't distinguish this tool from potential sibling tools like 'get_recent_emails' - both appear to retrieve email content, just with different scopes (specified vs recent).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'get_recent_emails' and 'send_email' available, there's no indication whether this tool is for retrieving full content of a specific known email versus browsing recent emails or composing new ones. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• setup_email_accountC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions automatic provider detection and server configuration, but lacks critical details: whether this creates a new account or configures an existing one, what permissions are required, if it's a read-only or destructive operation, potential rate limits, or what happens on success/failure. For a tool with no annotations and potential account setup implications, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: a single sentence in parentheses adds useful context about automatic detection. There's no wasted verbiage or redundancy, making it efficient for an agent to parse while conveying the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (account setup with 3 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like mutability, error handling, or return values, which are crucial for a tool that likely involves authentication and configuration. The description alone is insufficient for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters well-documented in the input schema (email, password, provider). The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain parameter interactions (e.g., how provider auto-detection works if omitted) or usage nuances. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '设置邮箱账号（自动识别邮箱类型并配置服务器）' translates to 'Set up email account (automatically identify email type and configure server).' It specifies the verb 'set up' and resource 'email account' with the additional functionality of automatic provider detection and server configuration. However, it doesn't explicitly differentiate from sibling tools like 'configure_email_server' or 'test_email_connection,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'configure_email_server' (which might be for manual configuration) or 'test_email_connection' (which might be for verification). There's no indication of prerequisites, such as needing an existing account or specific permissions, leaving the agent with no context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_email_connectionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool tests connections but doesn't explain what 'testing' entails—e.g., whether it performs authentication checks, network pings, or returns diagnostic details. It also omits critical behavioral traits like error handling, timeouts, or if it modifies server state (though 'test' implies read-only). For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient phrase ('测试邮箱服务器连接') that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool and front-loaded with the core action. Every part of the description earns its place by conveying essential intent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving server testing) and lack of annotations or output schema, the description is incomplete. It doesn't explain what the test does, what results to expect (e.g., success/failure indicators), or any dependencies (e.g., requiring prior server configuration). For a testing tool with no structured output documentation, the description should provide more context to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'testType' fully documented in the schema (including enum values and descriptions). The tool description adds no parameter-specific information beyond what the schema provides. According to the rules, when schema_description_coverage is high (>80%), the baseline score is 3 even with no param info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '测试邮箱服务器连接' clearly states the tool's purpose: to test email server connections. It uses a specific verb ('测试' meaning 'test') and resource ('邮箱服务器连接' meaning 'email server connection'), making the intent unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'configure_email_server' or 'setup_email_account', which might involve similar server interactions but for different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a configured server first), exclusions, or comparisons to siblings like 'configure_email_server' (for setup) or 'send_email' (for actual operations). Without such context, an agent might struggle to select this tool appropriately in scenarios involving email server interactions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_emailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does (get recent emails) but lacks critical behavioral details such as whether this is a read-only operation, if it requires authentication, how it handles errors, or what the return format looks like (e.g., list structure, pagination). This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a read operation with two parameters), no annotations, and no output schema, the description is incomplete. It adequately states the purpose but fails to provide necessary behavioral context (e.g., safety, authentication, return format) or usage guidelines, leaving gaps that could hinder an AI agent's ability to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters (limit and days) fully documented in the schema, including their types and default values. The description adds no additional parameter information beyond implying a three-day default for 'days', which is already covered in the schema. This meets the baseline score of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('获取' meaning 'get') and resource ('邮件列表' meaning 'email list'), and specifies a time scope ('最近三天' meaning 'last three days'). It distinguishes from siblings like get_email_content (which gets content of specific emails) and send_email (which sends emails). However, it doesn't explicitly contrast with list_supported_providers or other siblings, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving recent emails within a three-day window, but provides no explicit guidance on when to use this tool versus alternatives like get_email_content (for specific email details) or configure_email_server (for setup). No exclusions or prerequisites are mentioned, leaving usage context somewhat vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_supported_providersB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states what the tool does (lists providers) but doesn't disclose behavioral traits like whether this is a read-only operation, if it requires authentication, what format the output takes, or if there are rate limits. For a tool with zero annotation coverage, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese that directly states the tool's purpose with zero wasted words. It's appropriately sized for a simple listing tool and front-loads the core functionality immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema), the description is minimally adequate. It states the purpose but lacks context about output format, authentication needs, or when to use it relative to siblings. For a tool with no annotations and no output schema, the description should ideally provide more behavioral guidance to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the input (none required). The description doesn't need to add parameter information, and it doesn't contradict the schema. A baseline of 4 is appropriate for zero-parameter tools where the schema handles all input documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '列出支持的邮箱提供商' (List supported email providers) clearly states the verb (list) and resource (email providers). It distinguishes from siblings like configure_email_server or send_email by focusing on provider enumeration rather than configuration or sending operations. However, it doesn't explicitly differentiate from setup_email_account or test_email_connection which might involve providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether this should be called before setup_email_account), exclusions, or relationships to sibling tools like configure_email_server or setup_email_account. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.