Server Quality Checklist

Tool Scores

• search_usersC
  2.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but provides none. It doesn't indicate whether this is a read-only operation, what permissions might be required, whether results are paginated or limited, what happens with empty queries, or how results are sorted. For a search tool with zero annotation coverage, this represents a complete lack of behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - just two Chinese characters. While this represents efficient communication, it's arguably under-specified rather than appropriately concise. However, given the scoring framework, it earns points for being front-loaded with the core purpose and having zero wasted words, though it lacks the structure of a more complete description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (search operation with filtering), lack of annotations, and absence of an output schema, the description is insufficiently complete. It doesn't explain what the search returns, how results are formatted, whether there are limitations on search scope, or any error conditions. For a search tool that likely returns structured user data, this minimal description leaves critical gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters ('query' and 'role') having descriptions in the schema. The tool description adds no additional parameter information beyond what's already documented in the schema. According to the scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no parameter info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '搜索用户' (Search users) states the general purpose but is vague and tautological - it essentially restates the tool name 'search_users' in Chinese. It doesn't specify what kind of search this performs (exact match, partial, fuzzy), what user attributes are searched, or how results are returned. While it indicates the verb (search) and resource (users), it lacks the specificity needed for clear agent understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description doesn't mention any prerequisites, limitations, or comparison with sibling tools like 'add_user'. An agent wouldn't know if this is for finding existing users before adding new ones, or if there are specific scenarios where this search is appropriate versus other user-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateC
  2.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. The description '执行数学计算' doesn't reveal any behavioral traits such as side effects, error handling, performance characteristics, or output format. It's a minimal statement that fails to inform the agent about how the tool behaves beyond its basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single phrase '执行数学计算', which is front-loaded and wastes no words. It efficiently states the purpose without unnecessary elaboration, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description is incomplete. It doesn't address how parameters interact (e.g., using 'expression' vs. 'operation' with 'a' and 'b'), what the tool returns, or any error conditions. For a tool with multiple input options and no structured output, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear parameter descriptions (e.g., '第一个数字' for 'a', '数学表达式' for 'expression'). The description adds no additional meaning beyond what the schema provides, as it doesn't explain parameter relationships, constraints, or usage examples. Baseline score of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '执行数学计算' (perform mathematical calculation) states a general purpose but is vague. It doesn't specify what kind of calculations, what resources are involved, or how it differs from potential alternatives. While it indicates the domain (mathematics), it lacks specificity about scope or method.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, constraints, or comparison with sibling tools (e.g., add_user, create_todo, search_users), which are unrelated but highlight the lack of context. Usage is implied only by the general purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_todoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. '创建待办事项' implies a write operation (creation), but it doesn't disclose any behavioral traits such as permissions required, whether it's idempotent, error handling, or what happens on success/failure. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single phrase '创建待办事项', which is highly concise and front-loaded with the core action. It wastes no words, though it could benefit from slightly more context to improve clarity without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a creation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or any behavioral context, making it inadequate for an agent to fully understand how to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no meaning beyond what the input schema provides. Schema description coverage is 100%, with clear documentation for both parameters ('title' and 'userId'), so the baseline score of 3 is appropriate as the schema handles the parameter semantics adequately without additional value from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '创建待办事项' (Create todo) states a clear verb+resource combination, indicating it creates a todo item. However, it doesn't distinguish this tool from potential siblings like 'add_user' or 'calculate' beyond the obvious domain difference, and it lacks specificity about what kind of todo system or context it operates in.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are no explicit instructions on prerequisites, context, or comparisons to sibling tools like 'add_user' or 'search_users', leaving the agent to infer usage based solely on the tool name and parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_userC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. '添加新用户' implies a write/mutation operation, but it doesn't specify permissions required, whether the operation is idempotent, what happens on duplicate emails, or error conditions. This leaves significant gaps for a tool that creates users.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single phrase that directly states the tool's purpose without any unnecessary words. It's front-loaded and wastes no space, though this conciseness comes at the cost of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a user creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns, what permissions are needed, how conflicts are handled, or other important behavioral aspects that an agent needs to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters (email, name, role) well-documented in the schema itself. The description adds no additional parameter information beyond what's in the schema, so it meets the baseline for adequate coverage without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '添加新用户' (Add new user) clearly states the verb ('add') and resource ('user'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'search_users' or 'create_todo', which would require more specific context about what distinguishes user creation from other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_users' or other sibling tools. There's no mention of prerequisites, typical use cases, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.