Server Quality Checklist

Tool Scores

• get_departuresA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions real-time data inclusion and defaults to current date/time, but lacks disclosure on side effects (though likely read-only), rate limits, authentication, or data freshness. It partially covers behavior but is incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear main sentence and a structured Args section. Every sentence adds value, no fluff. It is front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, return value details are not needed. The description covers purpose, parameters, and included fields. It lacks error handling or edge cases (e.g., invalid station no departures), but is otherwise sufficient for a simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates. It explains eva_no with an example and reference to find_station, and specifies date/hour format (YYMMDD, HH) with defaults. This adds significant meaning beyond the plain schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'get' and resource 'timetable for a station within one hour slice', specifying real-time data like delays, platform changes, cancellations, and added trips. This distinguishes it from siblings like find_station (lookup) and get_live_changes/get_recent_changes (likely broader change tracking).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for station departures and references find_station for EVA lookup, but does not explicitly state when to use this tool versus alternatives like get_live_changes or get_recent_changes. No 'when not to use' guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_changesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the 2-minute time window, size comparison to sibling, and sorting by time. It does not mention idempotency, rate limits, or error behavior, but these are less critical for a simple polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence serves a purpose: the first states functionality, the second provides usage guidance and comparison, and the final section explains parameters. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters and an output schema, the description covers the essential purpose, usage, and parameter details. It omits error handling and output format, but the output schema compensates for the latter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It provides eva_no with an example and limit with default and sorting context, adding meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Get only the changes of the last 2 minutes for a station', providing a specific verb, resource, and time constraint. It also distinguishes from sibling get_live_changes by noting the size difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises 'use for frequent polling' and contrasts with get_live_changes, giving clear context. However, it does not explicitly state when not to use it or mention alternatives for historical data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_live_changesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses sorting by time, limit on results, and coverage of current day. However, does not mention what happens with no changes or the exact format, but these are minor given output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise, two short paragraphs. First sentence states purpose, second covers parameters. No unnecessary words, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose and parameters well. With existing output schema, return value explanation not needed. Could clarify difference from sibling get_recent_changes, but overall adequate for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage, so description compensates fully. Both parameters are explained with examples and meaning (eva_no: EVA number with example; limit: max stops sorted by time).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool retrieves real-time changes for a station, listing specific types (delays, platform changes, cancellations) and scope (current operating day). Differentiates from siblings like get_departures or get_recent_changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context (current operating day) but no explicit when-use or when-not-use compared to siblings. The sibling list is available, but the description does not directly guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_stationA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses search behavior (prefix, exact naming, umlaut issues, wildcard). Does not mention authentication or result limits, but output schema likely covers return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences and an example. Front-loaded with purpose, then parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with output schema, description covers all necessary aspects: what it does, what parameter accepts, and usage tips. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description fully compensates by explaining that 'pattern' can be station name, EVA number, or DS100 code, with prefix search and wildcard examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches German train stations and retrieves EVA numbers. The description distinguishes it from sibling tools which focus on live changes, departures, and recent changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage guidance including prefix search, exact naming recommendations, umlaut limitations, and wildcard support. However, does not explicitly state when not to use or compare to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.