Server Quality Checklist

Tool Scores

• get-ticketsC
  2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers none. It doesn't indicate whether this is a read-only operation, whether it requires authentication, what rate limits apply, what format the results come in, or any error conditions. This is inadequate for a tool with 10 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one short sentence) and front-loaded with the core purpose. However, this brevity comes at the cost of under-specification rather than efficient communication. Every word earns its place, but more words are needed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no output schema, no annotations), the description is completely inadequate. It doesn't explain what the tool returns, how to interpret results, error handling, or any behavioral characteristics. For a data query tool with multiple filtering and sorting options, this leaves the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 10 parameters (7 optional), the description provides no information about any parameters. It doesn't explain what 'trainFilterFlags', 'sortFlag', 'format', or other parameters mean, their expected values, or how they affect the query. The description fails to compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '查询12306余票信息' (Query 12306 ticket availability information) states the basic purpose (querying ticket availability from 12306) but is vague about scope and differentiation. It doesn't specify what kind of tickets (train tickets implied by 12306), what information is returned, or how it differs from sibling tools like 'get-interline-tickets'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, when to choose this over 'get-interline-tickets', or any context about appropriate use cases. The agent must infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-interline-ticketsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the limitation of returning only the first ten results, which is valuable context. However, it doesn't describe authentication requirements, rate limits, error conditions, or what the response format looks like. For a query tool with 12 parameters and no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with just two sentences in Chinese. The first sentence states the purpose, the second adds an important limitation. There's no wasted text, though it could be more structured with clearer separation of purpose versus constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 parameters, 3 required), 0% schema description coverage, no annotations, and no output schema, the description is insufficient. It provides basic purpose and one limitation but doesn't explain parameter usage, return values, or behavioral constraints. For a tool with this many undocumented parameters, the description should do much more to guide proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning none of the 12 parameters have descriptions in the schema. The tool description provides no information about any parameters - it doesn't explain what 'date', 'fromStation', 'toStation', 'middleStation', or any other parameters mean or how they should be formatted. The description fails to compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '查询12306中转余票信息' (query 12306 interline ticket availability). It specifies the resource (interline tickets) and verb (query), though it doesn't explicitly differentiate from sibling tools like 'get-tickets' which might query direct tickets. The purpose is specific but sibling differentiation is implicit rather than explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are several sibling tools for station lookups and ticket queries, but no indication of when this interline ticket tool is appropriate versus 'get-tickets' or other alternatives. The only usage context is the limitation '尚且只支持查询前十条' (currently only supports querying the first ten results).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-train-route-stationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description explains what information is returned (stations, arrival/departure times, dwell times) but doesn't cover important behavioral aspects like error handling, rate limits, authentication requirements, or data freshness. For a tool with 3 parameters and no annotation coverage, this leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that directly address purpose and usage. The first sentence clearly states what the tool does, and the second provides usage context. There's no unnecessary information or repetition, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters with 0% schema coverage, no annotations, and no output schema, the description is incomplete. It explains the purpose and basic usage but doesn't address parameter meanings, return format, error conditions, or other contextual details needed for effective tool invocation. The description should compensate more for the lack of structured documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning none of the 3 parameters (trainCode, departDate, format) have descriptions in the schema. The tool description doesn't mention any parameters or explain their meaning, format, or constraints. This leaves all parameters completely undocumented, which is inadequate given the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '查询特定列车车次在指定区间内的途径车站、到站时间、出发时间及停留时间等详细经停信息' (Query detailed stop information including stations, arrival times, departure times, and dwell times for a specific train within a specified interval). It specifies the verb (query) and resource (train route stations with detailed timing information). However, it doesn't explicitly differentiate from sibling tools like 'get-tickets' or 'get-interline-tickets', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage guidance: '当用户询问某趟具体列车的经停站时使用此接口' (Use this interface when users ask about the stops of a specific train). This gives a clear context for when to use the tool. However, it doesn't specify when NOT to use it or mention alternatives among the sibling tools, leaving some ambiguity about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-code-of-citysB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the tool's purpose and usage context, it doesn't describe important behavioral aspects: whether this is a read-only lookup (implied but not stated), what happens with invalid city names, whether there are rate limits or authentication requirements, or what the return format looks like. For a mapping tool with zero annotation coverage, this represents significant gaps in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently convey the core functionality and usage context. The first sentence states the purpose, and the second provides the specific usage scenario. There's no wasted language, and the information is front-loaded with the primary function. However, the structure could be slightly improved by explicitly separating purpose from usage guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter mapping function), no annotations, no output schema, and 0% schema description coverage, the description is minimally adequate but has clear gaps. It covers the basic purpose and usage context but lacks details about error handling, return format, parameter constraints, and differentiation from similar sibling tools. For a lookup tool that serves as a parameter preparation step for other interfaces, more complete documentation would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for its single parameter 'citys', so the description must compensate. It adds meaningful context by specifying that the parameter accepts '中文城市名' (Chinese city names) and that it's used when users provide city names as departure/destination points. However, it doesn't clarify whether this accepts single cities or multiple cities (despite the plural 'citys' parameter name), nor does it provide examples or format requirements. The description adds some value but doesn't fully compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: querying for station_code using Chinese city names. It specifies the verb '查询' (query) and resource 'station_code', and indicates the context of preparing parameters for departure/destination interfaces. However, it doesn't explicitly differentiate from sibling tools like 'get-station-code-by-names' or 'get-stations-code-in-city', which appears to serve similar mapping functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when users provide city names as departure or destination points to prepare station_code parameters for other interfaces. It specifies the input format (Chinese city names) and the intended downstream use. However, it doesn't explicitly state when NOT to use it or mention alternatives among the sibling tools, which limits the guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-by-telecodeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates this is a query/read operation (consistent with 'get' naming) and mentions it's for obtaining data, but doesn't specify authentication requirements, rate limits, error conditions, or response format. The debugging mention adds some context, but key behavioral aspects remain undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: first states core functionality, second provides usage context. Every phrase adds value without repetition. It's appropriately sized for a single-parameter lookup tool and front-loads the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with 1 parameter and no output schema, the description covers the basic purpose and usage context adequately. However, with no annotations and 0% schema coverage, it should ideally provide more behavioral details (response format, error handling) and parameter specifics. The debugging mention adds some completeness but doesn't fully address the gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (no parameter descriptions in schema), and the description mentions the station_telecode parameter and its purpose. However, it doesn't provide format details, examples, constraints, or explain what a telecode is. With 1 parameter and 0% schema coverage, the description adds basic meaning but doesn't fully compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: querying station details by station_telecode, listing specific information returned (name, pinyin, city). It distinguishes from siblings by focusing on telecode lookup rather than name-based or city-based searches. However, it doesn't explicitly contrast with specific sibling tools like get-station-code-by-names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: when station_telecode is known, for obtaining complete station data, or for special queries/debugging. It explicitly states it's less commonly triggered in general user conversations. However, it doesn't name specific alternative tools or provide explicit 'when-not-to-use' guidance beyond the general usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-stations-code-in-cityA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the tool returns '一个包含多个车站信息的列表' (a list containing multiple station information), it doesn't specify important behavioral aspects like whether the city name must be in Chinese characters, what happens if the city doesn't exist, whether there are rate limits, or what format the station information includes beyond name and code.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured Chinese sentence that efficiently communicates the tool's purpose, scope, and output format. Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter query tool with no annotations and no output schema, the description provides adequate basic information about what the tool does and what it returns. However, it lacks details about error conditions, response format specifics, and behavioral constraints that would be helpful for an AI agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It clearly explains that the 'city' parameter should be a '中文城市名' (Chinese city name), which adds crucial semantic information beyond what the bare schema provides. However, it doesn't specify format requirements (e.g., simplified vs traditional characters) or validation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('查询' - query), resource ('火车站的名称及其对应的 station_code'), and scope ('所有' - all stations in the city). It distinguishes itself from siblings like 'get-station-by-telecode' and 'get-station-code-by-names' by focusing on city-based retrieval of all stations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you need to get ALL stations in a Chinese city with their names and codes. However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools, though the distinction is implied through the different resource focus.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-station-code-by-namesA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the query behavior but lacks details about response format, error handling, rate limits, or authentication requirements. For a tool with no annotation coverage, this is insufficient behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first states the core functionality, the second provides usage context. No wasted words, well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter lookup), no annotations, and no output schema, the description is adequate but incomplete. It explains when to use the tool and what it does, but lacks details about return values, error cases, or operational constraints that would be needed for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only one parameter, the description adds significant value by explaining that stationNames should be specific Chinese station names used as departure/arrival locations. This clarifies the parameter's purpose beyond the bare schema, though it doesn't specify format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: querying station_code and station names using specific Chinese station names. It specifies the verb ('query'), resource ('station_code and station names'), and distinguishes from siblings by focusing on exact station names rather than city-based or telecode-based lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: when users provide specific station names as departure or arrival locations to prepare station_code parameters for other interfaces. It distinguishes from siblings by specifying it's for exact station names, not city-based or telecode-based searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-current-dateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it returns the current date (not time), uses Shanghai timezone, returns in yyyy-MM-dd format, and is designed for relative date parsing. The description doesn't mention error conditions or performance characteristics, but covers the essential behavior well for a simple utility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and well-structured. Two sentences cover purpose, timezone, format, and primary use case with zero wasted words. Every sentence earns its place by providing essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter tool with no annotations and no output schema, the description provides excellent context: purpose, timezone, format, and use case. It doesn't need to explain return values since the format is specified. The only minor gap is not explicitly stating this is a read-only operation, though that's implied by '获取' (get).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage. The description appropriately doesn't discuss parameters since none exist, but does mention the implicit 'current' aspect of the date retrieval. This exceeds the baseline of 3 for high schema coverage by providing context about what 'current' means in terms of timezone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '获取当前日期' (get current date) with specific details about timezone (Asia/Shanghai, UTC+8) and return format (yyyy-MM-dd). It distinguishes itself from sibling tools which are all about train/station/ticket queries, making this a unique date utility tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: '主要用于解析用户提到的相对日期（如“明天”、“下周三”），提供准确的日期输入' (mainly used to parse relative dates mentioned by users like 'tomorrow' or 'next Wednesday' to provide accurate date input). It doesn't explicitly mention when NOT to use it or name specific alternatives, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.