Server Quality Checklist

Tool Scores

• get-interline-ticketsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions a limitation ('只支持查询前十条' - only supports querying the first ten results), which is useful, but fails to describe other critical behaviors such as response format, error handling, authentication needs, or rate limits. For a query tool with six parameters, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences: one stating the purpose and one noting a limitation. It is front-loaded with the core function and avoids unnecessary details, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema), the description is incomplete. It lacks information on output format, error conditions, and how results are structured, which is critical for an AI agent to use this tool effectively. The limitation mentioned is helpful but insufficient for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning all parameters are well-documented in the input schema itself. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline of 3 without compensating or adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('查询' meaning 'query') and resource ('12306中转余票信息' meaning '12306 interline ticket availability information'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-tickets' (which likely queries direct tickets), leaving room for ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it '只支持查询前十条' (only supports querying the first ten results), which suggests a limitation on result scope. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get-tickets' or how it relates to sibling tools, relying on implicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-ticketsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. '查询' (query) implies a read-only operation, but the description doesn't explicitly state whether this requires authentication, has rate limits, returns paginated results, or what happens on errors. For a ticket query tool with zero annotation coverage, this represents significant behavioral gaps that could affect agent decision-making.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just 8 Chinese characters ('查询12306余票信息'), making it front-loaded and efficient with zero wasted words. Every character serves the purpose of identifying the tool's function. This represents optimal conciseness for a tool description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a ticket query system with 4 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what information is returned (ticket types, prices, availability status), how results are structured, or any behavioral constraints. The agent would need to infer too much about this tool's operation from the minimal description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides no parameter information beyond what's already in the schema. With 100% schema description coverage, all 4 parameters (date, fromStation, toStation, trainFilterFlags) are well-documented in the schema with format requirements, dependencies on other tools, and usage examples. The baseline score of 3 reflects that the schema does the heavy lifting while the description adds no additional parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '查询12306余票信息' clearly states the purpose as querying ticket availability from the 12306 system. It specifies the verb '查询' (query) and resource '余票信息' (ticket availability), making the tool's function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get-interline-tickets' which might handle different ticket types or routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context through the Chinese text, suggesting it's for checking ticket availability on China's 12306 railway system. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get-interline-tickets' or when not to use it. The input schema provides some usage hints (like calling other tools first for date/station codes), but these aren't part of the description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-code-of-citysA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the basic query functionality but lacks behavioral details such as error handling (e.g., what happens with invalid city names), rate limits, authentication requirements, or response format. For a tool with no annotation coverage, this leaves significant gaps in understanding how it behaves beyond the core purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the purpose, and the second provides usage context. Every sentence adds value without redundancy, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description covers the purpose and usage adequately but lacks details on behavior, error handling, and output format. For a simple query tool with one parameter, it's minimally viable but could be more complete by addressing missing behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'citys' fully documented in the schema (including format for single/multiple cities). The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '通过中文城市名查询代表该城市的 station_code' (query station_code representing the city via Chinese city name). It specifies the verb '查询' (query) and resource 'station_code', and mentions the context of departure/arrival locations. However, it doesn't explicitly differentiate from sibling tools like 'get-station-code-by-names' or 'get-stations-code-in-city', which appear related.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: '此接口主要用于在用户提供城市名作为出发地或到达地时，为接口准备 station_code 参数' (this interface is mainly used when users provide city names as departure or arrival locations to prepare station_code parameters for other interfaces). This indicates when to use it (for city-to-code mapping in travel contexts) but doesn't explicitly mention when not to use it or name specific alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-by-telecodeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes the query behavior (lookup by telecode) and typical use cases, but doesn't disclose important behavioral traits like error handling, response format, whether it's read-only, performance characteristics, or authentication requirements. The description adds some context about usage frequency but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: first states the core functionality, second provides usage context. Every phrase adds value with no redundancy. It's appropriately sized for a single-parameter lookup tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with 100% schema coverage but no annotations or output schema, the description provides adequate purpose and usage context. However, it lacks details about return values, error conditions, and operational constraints that would be helpful given the absence of structured output documentation. The description is complete enough for basic understanding but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter. The description mentions 'station_telecode' but adds no additional semantic context beyond what's in the schema (3-letter code). The baseline is 3 when schema does the heavy lifting, and the description doesn't compensate with extra parameter insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: query station details by station_telecode, listing specific information returned (name, pinyin, city). It distinguishes from siblings by focusing on telecode lookup rather than name-based or city-based searches. However, it doesn't explicitly name alternative tools for different query types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool: 'when telecode is known' and for 'special queries/debugging.' It explicitly states this tool is 'less commonly triggered directly in user conversations,' which helps guide usage frequency. However, it doesn't name specific alternative tools for different scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-code-by-namesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool is for querying ('查询'), implying a read-only operation, and mentions its primary use case for preparing parameters. However, it lacks details on error handling, rate limits, authentication needs, or what happens with invalid inputs, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded and efficient. The first sentence states the core purpose, and the second adds usage context without redundancy. Every sentence earns its place by contributing essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is somewhat complete but has gaps. It explains the purpose and usage context adequately, but without annotations or output schema, it should ideally provide more behavioral details (e.g., response format, error cases) to fully compensate for the lack of structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter 'stationNames'. The description adds minimal value beyond the schema by reinforcing that it accepts Chinese station names and is for preparing parameters, but does not provide additional syntax, format details, or examples beyond what the schema specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('查询' - query) and resources ('station_code' and station name), explicitly distinguishing it from sibling tools by focusing on querying by Chinese station names rather than telecodes, cities, or other parameters. It directly addresses what the tool does without tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('在用户提供具体车站名作为出发地或到达地时' - when users provide specific station names as departure or arrival points), but does not explicitly mention when not to use it or name specific alternatives among the sibling tools. The guidance is helpful but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-train-route-stationsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly describes what the tool returns (stop stations, arrival/departure times, dwell times) but doesn't mention error conditions, rate limits, authentication needs, or response format. For a query tool with no annotation coverage, this provides basic behavioral information but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences with zero waste. First sentence defines the tool's purpose and scope, second sentence provides clear usage guidance. Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with 4 required parameters, 100% schema coverage, but no annotations and no output schema, the description provides adequate purpose and usage context. However, it doesn't describe the return format or structure, which would be helpful given the absence of an output schema. The description is complete enough for basic understanding but lacks output details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all 4 parameters. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. Baseline 3 is appropriate when the schema does all the parameter documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('查询' - query) and resource ('特定列车车次在指定区间内的途径车站、到站时间、出发时间及停留时间等详细经停信息' - detailed stop information for specific train routes). It distinguishes from siblings by specifying it's for '具体列车的经停站' (specific train stop stations), unlike general ticket or station lookup tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states '当用户询问某趟具体列车的经停站时使用此接口' (use this interface when users ask about specific train stop stations), providing clear when-to-use guidance. It distinguishes from alternatives by specifying it's for detailed stop information rather than ticket availability or station codes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-stations-code-in-cityA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the output format ('结果是一个包含多个车站信息的列表' - result is a list containing multiple station information) and scope ('所有' - all stations), but lacks details on error handling, rate limits, authentication needs, or whether the query is case-sensitive for Chinese city names. It adequately describes core behavior but misses operational nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first explains what the tool does and its output, the second provides usage guidelines. Every sentence adds value without redundancy, and it's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no annotations, no output schema), the description is mostly complete: it covers purpose, usage, output format, and parameter context. However, it lacks details on error cases (e.g., invalid city names) and exact return structure, which would be helpful since there's no output schema. It's sufficient but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by emphasizing the parameter must be a '中文城市名' (Chinese city name) and providing context that it's used to query '所有' (all) stations, which clarifies the parameter's role beyond the schema's basic type and example. However, it doesn't add syntax or format details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('查询' - query) and resource ('火车站的名称及其对应的 station_code'), explicitly distinguishes scope ('所有' - all stations in the city), and differentiates from siblings like 'get-station-by-telecode' (which uses telecode) and 'get-station-code-by-names' (which uses station names).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('当用户想了解一个城市有哪些火车站，或者不确定具体从哪个车站出发/到达时' - when users want to know which stations a city has or are unsure about departure/arrival stations), providing clear context for its application without needing to mention specific alternatives since the sibling tools serve different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-current-dateA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: timezone specificity (Asia/Shanghai), return format (yyyy-MM-dd), and the tool's role in date parsing workflows. It doesn't mention performance characteristics or error conditions, but covers the essential operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences with zero waste. The first sentence states what the tool does (purpose, timezone, format). The second sentence explains when to use it (usage context). Every element serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter tool with no output schema, the description provides excellent context about timezone, format, and usage scenarios. It doesn't need to explain return values since the format is specified. The only minor gap is lack of explicit mention about what happens on errors or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on the tool's output and usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '获取当前日期' (get current date) with specific details about timezone (Asia/Shanghai, UTC+8) and return format (yyyy-MM-dd). It distinguishes itself from sibling tools by focusing on date retrieval rather than train/ticket/station operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: '主要用于解析用户提到的相对日期（如“明天”、“下周三”），为其他需要日期的接口提供准确的日期输入' (mainly used to parse relative dates mentioned by users like 'tomorrow' or 'next Wednesday', providing accurate date input for other interfaces that need dates). This gives clear context about its intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.