Server Quality Checklist

Tool Scores

• get_holiday_statsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does (retrieve holiday statistics) but lacks details on behavioral traits such as whether it's read-only, any rate limits, authentication needs, or what the output format might be (e.g., counts, lists, or aggregated data). This leaves significant gaps for an agent to understand how to handle the tool effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every part of the sentence contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a tool that retrieves statistical information, the description is incomplete. With no annotations and no output schema, it fails to explain what '統計資訊' (statistics information) entails—such as the type of data returned (e.g., counts, percentages, or detailed breakdowns). This lack of context makes it harder for an agent to use the tool correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for both parameters ('year' and 'month') including constraints. The description adds minimal value beyond the schema by mentioning '指定年份或年月' (specified year or year-month), which aligns with the schema but doesn't provide additional semantic context or usage examples. Baseline 3 is appropriate as the schema handles most of the parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('獲取' meaning 'get' or 'retrieve') and the resource ('台灣假期統計資訊' meaning 'Taiwan holiday statistics information'), specifying it's for a given year or year-month. However, it doesn't explicitly differentiate from sibling tools like 'check_holiday' or 'get_holidays_in_range', which might have overlapping purposes but different scopes or outputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools 'check_holiday' or 'get_holidays_in_range'. It mentions the parameters (year or year-month) but doesn't clarify the context or alternatives, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_holidayB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't describe how it behaves: it doesn't mention data sources, accuracy, update frequency, error handling, or response format. For a tool with no annotations, this leaves significant gaps in understanding its operational characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's purpose without any unnecessary words. It's appropriately sized and front-loaded, with every word earning its place. No structural issues or verbosity detract from its clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage, the description is minimally adequate. However, with no output schema and no annotations, it doesn't explain what the return value looks like (e.g., boolean, holiday name, or error messages). For a simple lookup tool, this is acceptable but leaves room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'date' parameter fully documented in the schema itself (format, pattern). The description adds no additional parameter semantics beyond what's already in the schema. According to the rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '檢查指定日期是否為台灣假期' (Check if a specified date is a Taiwan holiday). It uses a specific verb ('檢查' - check) and resource ('台灣假期' - Taiwan holidays). However, it doesn't explicitly distinguish itself from sibling tools like 'get_holidays_in_range' or 'get_holiday_stats', which reduces it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or suggest scenarios where this single-date check would be preferred over range-based queries or statistical tools. The agent must infer usage from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_holidays_in_rangeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states what the tool does but lacks behavioral details: no mention of permissions needed, rate limits, whether it returns structured data or raw text, error handling, or pagination. For a read operation with no annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise with a single sentence that directly states the purpose. No wasted words or redundant information. It's front-loaded and efficiently communicates the core functionality without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (date-range query), no annotations, and no output schema, the description is minimally adequate. It covers the basic purpose but lacks details on output format, error cases, or behavioral traits. It meets the bare minimum for a read operation but doesn't provide enough context for robust agent use without additional assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('start_date', 'end_date') fully documented in the schema regarding format and patterns. The description adds no additional parameter semantics beyond implying a date range, which is already clear from the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('獲取' - get/fetch) and resource ('台灣假期' - Taiwan holidays) with scope ('指定日期範圍內' - within a specified date range). It distinguishes from sibling 'check_holiday' (likely checks a single date) and 'get_holiday_stats' (likely provides statistics), though not explicitly. The purpose is specific but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'check_holiday' or 'get_holiday_stats'. The description implies usage for fetching multiple holidays in a range, but doesn't state exclusions or prerequisites. It's minimal, relying on inference from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.