Server Quality Checklist

Tool Scores

• service_alertsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It states that the tool returns active alerts with headlines, descriptions, affected stations, and date ranges, but it does not mention pagination, sorting, or any potential side effects. For a read-only fetch, this is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, starting with a clear purpose, then a brief summary of return fields, and ending with parameter details. The bitmask list is lengthy but necessary for usability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Because an output schema exists, the description need not detail return values, but it still summarizes them helpfully. The parameter semantics and purpose give enough context to invoke the tool correctly, though there is some room to mention behavior like result ordering or time windows.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The Args section fully explains both parameters despite the schema having 0% description coverage. It defines max_alerts as a limit and provides a complete product_filter bitmask with numeric values, which is essential for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Fetch current OeBB service alerts and disruptions,' using a specific verb and resource. This clearly distinguishes it from the sibling tools, which handle station search, station boards, and trip search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies when to use the tool, but there is no explicit guidance about when not to use it or how it compares to alternatives. The sibling tools are semantically distinct, so the intended context is mostly clear without further explanation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• station_boardA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It states that data is live and includes return fields, and that station name is auto-resolved. However, it does not specify behavior when both station_id and station_name are provided, error handling, or any side effects (though it is clearly read-only). This is adequate but not deeply transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening sentence and a parameter list. It is reasonably concise, though some redundancy exists between the opening guidance and the parameter details. It remains easy to scan and front-loads the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (so return format is covered), the description covers all essential usage aspects: station identification, board type, and result count. It could be more complete by addressing ambiguity (e.g., what if both station_id and station_name are given) or error cases, but overall it is solid for a read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides zero description for parameters (0% coverage), so the description fully compensates by explaining each parameter with examples and default behavior. It clarifies board_type values, station_id format, and max_journeys semantics, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches live departures or arrivals at an OeBB station, with a specific verb (fetch) and resource (station board). It distinguishes itself from siblings like search_station and trip_search by focusing on real-time station-level data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use this tool (live departures/arrivals) and explains how to specify the station via ID or name. However, it does not explicitly mention alternatives or when not to use it, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trip_searchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden, and it does well by stating what the tool returns ('departure/arrival times, duration, changes, and legs') and mentioning defaults for time/time_mode. It does not discuss error cases or rate limits, but for a search tool, this is adequate and transparent about core behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively long but well-structured: an executive summary, a usage note, a return description, and a numbered Args list. Every sentence adds value, though a tiny bit of redundancy exists (e.g., 'Provide station IDs or names' repeated in Args). It earns a 4, not a 5, due to slight verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 params, no schema descriptions, no annotations), this description is comprehensive. It covers all parameters, defaults, supported time modes, and returned data. It also hints at the relationship between IDs and names. However, it does not explain what happens if both an ID and name are provided (which takes precedence) or mention sibling tools for alternative use cases, so it is not a perfect 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates by explaining every parameter with examples ('Wien Hbf', '2026-04-15T08:00:00'), clarifying that station IDs or names can be used, and specifying defaults and modes. This goes far beyond the raw schema and gives the AI agent everything needed to correctly fill parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search train connections between two OeBB stations.' This clearly distinguishes it from siblings like station_board (which likely shows departures for a single station) or search_station (which finds stations). The scope is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context by mentioning 'Supports future trip planning via time/time_mode parameters,' implying when to use this for planning versus real-time lookups. However, it does not explicitly name sibling tools or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_stationA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states that it returns matching stations with IDs, coordinates, and types, and includes the max_results default. It does not mention limitations like rate limits or search specificity, but for a simple read-only search, it covers the essential behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a clear introduction, usage note, return summary, and parameter list. Every sentence serves a purpose, and it is front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, the presence of an output schema, and the description covering purpose, usage, returns, and parameters, the description is complete enough for an agent to correctly select and invoke the tool. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must fully compensate. It provides detailed Args explanations, including an example for query and the default for max_results, adding meaningful context beyond the bare schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches OeBB stations by name, with a specific verb and resource. It also distinguishes itself from sibling tools (station_board, trip_search, service_alerts) by highlighting its role in retrieving station IDs for other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool to find station IDs needed for other tools, providing clear context for when to use it. However, it does not explicitly mention exclusions or alternative tools, so it misses the top score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.