Server Quality Checklist

Tool Scores

• search_trialsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the search functionality and return format (trial list), it lacks important behavioral details: whether this is a read-only operation (implied but not stated), any rate limits, authentication requirements, pagination behavior beyond max_results, or what happens when no results are found. The description provides basic functional information but misses key operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise - a single sentence that efficiently communicates the core functionality. It's front-loaded with the main action (searching ChiCTR clinical trials) followed by the search criteria and return format. There's no wasted language, though it could potentially benefit from slightly more detail given the lack of annotations and output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with 4 parameters, no annotations, and no output schema, the description is incomplete. While it covers the basic 'what', it lacks important context: what the returned trial list contains, how results are ordered, whether all parameters are optional (implied but not stated), error conditions, or any limitations of the search functionality. The absence of output schema means the description should ideally provide some indication of return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions the three main search parameters (title keywords, registration number, year) which aligns with the input schema's properties. However, with 100% schema description coverage, the schema already provides excellent parameter documentation including examples and defaults. The description adds minimal value beyond what's in the schema - it confirms the parameter purposes but doesn't provide additional context about parameter interactions, search logic, or edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching ChiCTR clinical trials with specific search criteria (title keywords, registration number, year) and returning a list of trials. It uses specific verbs ('搜索' - search, '返回' - return) and identifies the resource (ChiCTR临床试验). However, it doesn't explicitly differentiate from sibling tools like 'get_trial_detail' which likely retrieves detailed information about a specific trial rather than searching multiple trials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While it mentions what the tool does, it doesn't indicate when it's appropriate to use search_trials versus get_trial_detail (which presumably gets details for a specific trial) or other siblings. There's no mention of prerequisites, limitations, or typical use cases beyond the basic functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_cacheB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. '清除所有缓存数据' implies a destructive mutation (clearing all cache), but it doesn't specify whether this action is reversible, requires authentication, has side effects (e.g., temporary slowdown), or returns confirmation. For a mutation tool with zero annotation coverage, this is a significant gap in safety and operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('清除所有缓存数据') with zero waste. It's front-loaded with the core action and resource, making it immediately understandable. No extraneous words or structural issues are present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive mutation with no parameters) and lack of annotations/output schema, the description is incomplete. It doesn't cover behavioral aspects like permissions needed, confirmation process, or impact on system performance. For a tool that clears all cache data, this minimal description leaves critical operational gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100% (empty schema). The description doesn't need to explain parameters, as there are none. It appropriately focuses on the tool's action without redundant parameter details, meeting the baseline for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '清除所有缓存数据' (Clear all cache data) clearly states the tool's action and resource with a specific verb ('清除' - clear) and target ('所有缓存数据' - all cache data). It distinguishes from sibling tools like 'get_cache_stats' (read-only) and 'search_trials' (different resource), though it doesn't explicitly mention these distinctions. The purpose is unambiguous but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., admin permissions), consequences (e.g., performance impact), or when to avoid it (e.g., during peak usage). With siblings like 'get_cache_stats' for monitoring, the description fails to help the agent choose between read and write operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cache_statsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions what statistics are retrieved (hit rates for search and detail caches), it doesn't disclose important behavioral traits like whether this requires authentication, rate limits, what format the statistics are returned in, or whether this is a read-only operation. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the purpose without unnecessary elaboration. It's appropriately sized for a zero-parameter tool that retrieves statistics, though it could be slightly more structured by explicitly stating it's a read operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no annotations, and no output schema, the description provides basic purpose information but lacks important context. For a statistics retrieval tool, it should ideally mention the format of returned data, whether it's real-time or cached statistics, and any authentication requirements. The description is minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and the schema already fully documents this (empty object).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '获取缓存统计信息' (get cache statistics) with specific mention of '搜索缓存和详情缓存的命中率等' (hit rates for search cache and detail cache). It uses a specific verb ('获取' - get) and resource ('缓存统计信息' - cache statistics), though it doesn't explicitly distinguish from sibling tools like clear_cache.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when this tool should be used, when it shouldn't, or how it relates to sibling tools like clear_cache, get_trial_detail, or search_trials.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_trial_detailB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a query operation, implying it's likely read-only and non-destructive, but doesn't explicitly confirm this or describe other traits like error handling, rate limits, authentication needs, or what constitutes '完整详细信息' (complete detailed information) in the response. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized for a simple lookup tool and front-loaded with the core functionality, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one required parameter, no nested objects) and high schema coverage, the description is adequate but has clear gaps. It lacks output schema information, so the agent doesn't know what '完整详细信息' entails in the response. Combined with no annotations and minimal behavioral context, this leaves the description incomplete for optimal agent use, though it meets minimum viability for a simple query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantic context beyond the input schema. It mentions the parameter '注册号' (registration number), which aligns with the schema's 'registration_number' property. However, with 100% schema description coverage (the schema already documents the parameter as '临床试验注册号，如 'ChiCTR2400084905''), the description doesn't provide additional details like format examples or usage nuances. This meets the baseline score of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '根据注册号查询临床试验的完整详细信息' (Query complete detailed information of clinical trials based on registration number). It specifies the verb ('查询' - query) and resource ('临床试验的完整详细信息' - complete detailed information of clinical trials), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'search_trials', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'search_trials' or clarify scenarios where this tool is preferred (e.g., for looking up specific trials by exact registration number versus broader searches). There's also no information about prerequisites or exclusions, leaving usage context implied at best.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.