Server Quality Checklist

Tool Scores

• get_stationboardA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool returns live data with times, platforms, destinations, and delays. However, it does not mention any potential rate limits, authentication needs, or side effects, though the tool is read-only and safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a brief intro, bullet-pointed use cases, and a summary sentence. It is well-structured, front-loaded, and contains no extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 5 parameters and existence of an output schema, the description covers the main functionality and return format. It lacks details on error handling or constraints (e.g., valid stations), but is adequate for the tool's purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains transport_types filtering and implies mode via 'departures/arrivals', but does not explicitly describe station, limit, or datetime parameters, leaving significant gaps in parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks departures/arrivals at Swiss stations with specific use cases. It uses a strong verb ('Check') and resource ('stationboard'), and the sibling tools (plan_journey, search_locations, get_booking_link) are distinct, so differentiation is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases in bullet points, making it clear when to use this tool. However, it does not explicitly state when not to use it or directly name alternatives like plan_journey for journey planning, leaving a slight gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_locationsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden. It does not declare whether the tool is read-only or describe any side effects, though it implies a query-like operation. The mention of 'returns station IDs, coordinates, and relevance scores' is helpful, but lacks explicit behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, using a short paragraph with bullet points. Every sentence adds value, and the purpose is front-loaded. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values are covered. The description addresses key aspects: search by name or coordinates, and resolving ambiguity. However, it omits explanation of the 'type' parameter and does not mention any result limits or pagination, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds some meaning by mapping use cases to parameters: 'query' for name, 'latitude'/'longitude' for GPS. However, the 'type' parameter is not described, leaving a gap. The description provides partial compensation, but not full coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find Swiss train stations, bus stops, and other transport locations.' It distinguishes from siblings like get_booking_link, get_stationboard, and plan_journey, which handle different tasks. The verb 'search' and resource 'locations' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: look up station name, find near GPS, resolve ambiguous names. It implies usage before planning a journey, but does not explicitly state when not to use or mention alternatives. However, it gives sufficient context for appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_booking_linkA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It transparently states that the tool returns a 'ready-to-click URL' with search pre-filled. While it could mention error behavior or rate limits, the core behavioral trait is clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with three bullet points and a summary sentence. Every sentence adds value, no fluff, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters (2 required) and an output schema. The description covers the overall purpose and return value well but omits details on optional parameters and input validation. For a straightforward URL-generation tool, this is mostly adequate but could be slightly more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the tool description does not explain any parameters. Though the description mentions 'specific journey', it does not detail what each parameter (from_station, to_station, date, time, is_arrival_time) means or how to format them, leaving the agent without guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a direct booking link for a specific journey. It lists concrete use cases like buying a ticket or sharing a timetable link, effectively distinguishing it from siblings like 'plan_journey' or 'get_stationboard'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly gives three scenarios for using the tool ('buy a ticket', 'get a link', 'share a timetable link'), which guides the agent on when to invoke it. However, it does not explicitly state when NOT to use it or mention alternatives among siblings, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_journeyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It discloses that the tool returns real-time schedules with platform numbers, delays, transfers, and occupancy levels. It also mentions calling multiple times for full day itinerary. Does not mention destructive actions or authentication needs, but overall adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences plus bullet-like usage list. Information is front-loaded with purpose, then usage scenarios, then return details. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is an output schema, description does not need to explain returns. It covers usage, key parameters, and behavior. However, with 8 parameters and only partial description, it could be more thorough. Still adequate for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. It explains only some parameters: 'from_station', 'to_station', 'via', and 'is_arrival_time'. It does not explain 'date', 'time', 'transport_types', or 'limit'. This partial coverage provides some value but leaves gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it plans a journey through Switzerland by specific transport types. It lists explicit use cases (from A to B, multi-stop, etc.) and the verb 'plan' with resource 'journey' is specific. Distinguishes from siblings like get_booking_link, get_stationboard, search_locations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this whenever someone wants to:' and lists four scenarios. It gives clear context but does not mention when not to use it or compare with alternatives. Parenthetical examples like 'use `via` for intermediate stops' provide guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.