Server Quality Checklist

Tool Scores

• zentao_bug_getC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('Get'), but doesn't mention authentication needs, rate limits, error handling, or what the return format looks like (e.g., JSON structure). For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of a single, efficient sentence with bilingual phrasing. Every word earns its place, and there's no wasted verbiage, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that retrieves details. It doesn't explain what 'bug details' include (e.g., fields returned), error cases, or dependencies. For a read operation with no structured output information, this leaves the agent under-informed about the tool's full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'id' fully documented as 'Bug ID (required).' The description adds minimal value beyond this by reiterating 'by bug ID', but doesn't provide additional context like valid ID ranges or examples. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get bug details by bug ID.' It specifies the verb ('Get') and resource ('bug details'), and the bilingual phrasing adds clarity. However, it doesn't explicitly distinguish this tool from its sibling 'zentao_bugs_list', which might also retrieve bug details in a list format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'zentao_bugs_list' or 'zentao_bugs_mine'. It implies usage by stating 'by bug ID', but lacks explicit context, prerequisites, or exclusions, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zentao_bugs_listA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly indicates this is a read operation ('List bugs'), but doesn't disclose behavioral traits like pagination behavior (implied by page/limit parameters), rate limits, authentication requirements, or what the return format looks like. It adds some context about the prerequisite but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the purpose and usage context, the second provides the prerequisite. Every sentence adds value with no redundant information, and it's appropriately front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list operation with 3 parameters and no output schema, the description provides adequate purpose and usage guidance but lacks details about return format, pagination behavior, or error conditions. The absence of annotations means the description should do more to cover behavioral aspects, though it successfully addresses the core use case and prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds minimal value beyond the schema by mentioning the product ID prerequisite, but doesn't provide additional semantic context about parameter usage or relationships. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List bugs') and resource ('for a specific product in ZenTao'), distinguishing it from sibling tools like zentao_bug_get (single bug) and zentao_bugs_mine (user-specific bugs). It provides both English and Chinese terminology for clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('when user asks to see/view/show bugs or wants to check issues for a product') and provides alternative phrasing in both English and Chinese. It also mentions a prerequisite ('Requires product ID which can be obtained from zentao_products_list') and implicitly distinguishes from siblings by focusing on product-specific listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zentao_bugs_mineA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the default scope ('assigned') and includes examples of user queries, which adds some behavioral context. However, it lacks details on permissions, rate limits, pagination behavior, or what the response looks like (e.g., format, error handling). For a tool with 8 parameters and no annotations, this is a moderate gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it states the purpose in the first sentence, provides usage guidelines in the second, and includes specific query examples. Every sentence earns its place with no wasted words, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no output schema, no annotations), the description is adequate but has gaps. It covers purpose and usage well, but lacks behavioral details like response format, error cases, or performance considerations. With no output schema, it should ideally hint at return values, but it doesn't, leaving some context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal parameter semantics beyond the schema—it implies the 'scope' parameter with 'Default scope is assigned' but doesn't explain other parameters like 'account' or 'status'. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List my bugs (我的Bug) by assignment or creator.' It specifies the verb ('List'), resource ('my bugs'), and scope ('by assignment or creator'), and distinguishes it from siblings by focusing on personal bug lists rather than general bug listing or product listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Default scope is assigned. Use when user asks for 'my bugs', '我的bug', '分配给我', or personal bug list.' It specifies when to use the tool (personal bug queries) and implies alternatives (e.g., zentao_bugs_list for non-personal lists), though it doesn't name siblings directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zentao_products_listA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return format ('product information including ID, name, and bug counts') which is helpful, but doesn't address important behavioral aspects like pagination behavior (implied by parameters but not explained), error conditions, or authentication requirements. The description adds some value but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence states the purpose, and the second provides usage guidance and return format information. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with no annotations and no output schema, the description provides good context about purpose, usage, and return format. However, it doesn't fully compensate for the lack of output schema by detailing the exact structure of returned product information or addressing pagination behavior. Given the tool's relative simplicity, it's mostly complete but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description doesn't add any parameter-specific information beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all products from ZenTao'), and distinguishes this tool from its siblings by specifying it's for getting product IDs before querying bugs. It explicitly mentions the sibling tools are for bugs, while this is for products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('to get product IDs before querying bugs') and implicitly suggests alternatives by mentioning sibling bug-related tools. It establishes a clear workflow context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.