Server Quality Checklist

Tool Scores

• get_timetableB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions retrieval and formatting but does not clarify if it is read-only, what happens if the station does not exist, or any other important side effects or constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the primary action, the second elaborates on the content. No redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description sufficiently covers the tool's functionality. However, it lacks usage guidance and differentiation from sibling tools, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the only parameter (station_number) with a description including an example. The tool description does not add extra meaning beyond what the schema already provides, so it meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves formatted timetable information for all routes at a station, using a specific verb and resource. However, it does not explicitly distinguish itself from sibling tools like get_approach_for_route, which fetch more specific data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., getting approach info for a specific route). It only describes what it does, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_approach_for_routeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions both historical and current data but does not disclose read-only nature, rate limits, data freshness, or other behavioral traits. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, first stating the core function and second adding context. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is fairly complete for a simple tool with one parameter and an output schema. It covers both historical and current aspects but could mention limitations or expected usage scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for route_code. The tool description adds no additional parameter semantics beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets real-time approach and position information for a route, distinguishing it from the sibling tool get_approach_for_station. It also specifies that it provides both historical and current data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for route-level data but does not explicitly provide when to use this tool versus the sibling get_approach_for_station or other tools. It lacks guidance on exclusions or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_approach_for_stationA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description effectively discloses key behavioral traits: it filters out routes without recent activity, sorts by proximity, and includes a URL. This goes beyond basic functionality and helps an agent understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences, front-loading the main purpose and then providing clear details about filtering and sorting. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple parameter and the presence of an output schema, the description covers all essential aspects: what it does, what is filtered, sorting order, and the inclusion of a URL. It is fully complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear parameter description. The tool description does not add additional meaning beyond the schema's existing description, so it meets the baseline without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets real-time bus approach information for all routes at a station, distinguishing it from siblings like get_approach_for_route (specific route) and get_timetable (timetable). The verb 'Get' and resource 'approach for station' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use (to get approach info for all routes at a station) and what the output contains. However, it does not explicitly exclude use cases or mention alternatives, leaving some ambiguity for agents to infer from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_station_numberA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the fuzzy matching behavior and the fallback from exact match, which is transparent. It does not mention error handling or read-only nature, but for a simple lookup, the disclosed behavior is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, consisting of two short sentences. The first sentence states the purpose and method, and the second elaborates on the algorithm. There are no superfluous words, and the information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and an output schema (though not shown), the description is fairly complete. It explains the input, the algorithm, and the purpose. It does not mention the output format or error handling, but the output schema likely covers the return value. For its simplicity, it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for the single parameter, with a description. The tool description adds details about the matching algorithm beyond the schema, providing context on how the parameter is used. This adds value beyond the schema's built-in description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to get a station number from a station name using fuzzy matching. It specifies the verb 'get', the resource 'station number', and adds detail about the matching logic. It distinguishes itself from siblings by being a lookup tool, while siblings are for routes and timetables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool: when you have a station name and need its number. It provides context about the matching algorithm (exact then fuzzy with 60% threshold), which guides usage. However, it does not explicitly state when not to use or mention alternatives, but the sibling tools are clearly different, so this is acceptable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.