Server Quality Checklist

Tool Scores

• findStationsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the search functionality and return type (list of stations) but lacks details on permissions, rate limits, error handling, or whether it's a read-only operation. For a search tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in three sentences: it states the action, explains the parameter, and describes the output. Every sentence adds value without redundancy, making it efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with one parameter) and no annotations or output schema, the description is minimally adequate. It covers the basic purpose and parameter but lacks details on behavioral traits, usage context, and output format. It meets the minimum viable standard but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already fully documents the single parameter 'pattern'. The description adds minimal value by mentioning it can be a station name or EVA number, which is already implied in the schema's example. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching a directory of railway stations using a search pattern (station name or EVA number) and returning a matching list. It specifies the resource (station directory) and verb (search), but doesn't explicitly differentiate from sibling tools like timetable or change queries, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description explains what it does but doesn't mention when it's appropriate compared to sibling tools (getCurrentTimetable, getPlannedTimetable, getRecentChanges) or any other search methods. There's no context about prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getPlannedTimetableB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it indicates this is a read operation for retrieving data, it doesn't disclose important behavioral traits such as whether the tool requires authentication, has rate limits, what format the returned data takes, or whether it supports pagination for large result sets. The description adds minimal behavioral context beyond the basic retrieval function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that each serve a purpose: the first states the core functionality, the second provides usage context. It's front-loaded with the main purpose and avoids unnecessary elaboration. While efficient, it could potentially be slightly more concise by combining the two sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with 3 well-documented parameters and no output schema, the description provides adequate but incomplete context. It covers the basic purpose and usage scenario but lacks important details about the return format, error conditions, and behavioral constraints. Without annotations or output schema, the description should do more to help an agent understand what to expect from the tool's execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage with clear parameter documentation including examples and format specifications. The description doesn't add any meaningful parameter semantics beyond what's already in the schema - it merely restates that parameters are for station, date, and hour without providing additional context or clarification. With complete schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving planned timetable data for a specified train station, date, and hour. It uses specific verbs ('holt...ein' - fetches/retrieves) and identifies the resource ('Fahrplandaten' - timetable data). However, it doesn't explicitly differentiate from sibling tools like getCurrentTimetable, which appears to serve a similar function but for current rather than planned data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by stating the tool is useful for planning schedules in advance and obtaining information about future train connections. However, it doesn't explicitly state when to use this tool versus alternatives like getCurrentTimetable or findStations, nor does it mention any prerequisites or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getRecentChangesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions real-time updates and lists types of changes (delays, track changes, cancellations), but doesn't address critical aspects like rate limits, authentication requirements, data freshness guarantees, or error conditions. For a real-time data tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently convey the tool's purpose and scope. The first sentence states the core function, and the second elaborates on what's included and the real-time nature. There's minimal wasted verbiage, though it could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (real-time operational data), no annotations, and no output schema, the description provides adequate basic context but lacks completeness. It covers what data is retrieved but doesn't address format, pagination, error handling, or limitations. The absence of output schema means the description should ideally explain return values, which it doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% with the single parameter 'evaNo' well-documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Ermittelt die neuesten Fahrplanänderungen für eine spezifische Bahnhofsstation' (retrieves the latest schedule changes for a specific train station). It specifies the verb 'ermittelt' (retrieves) and resource 'Fahrplanänderungen' (schedule changes), and distinguishes from siblings by focusing on real-time operational adjustments rather than station lookup or timetable retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'Echtzeit aktualisiert werden' (updated in real-time), suggesting this tool is for current operational changes rather than planned timetables. However, it doesn't explicitly state when to use this tool versus alternatives like getCurrentTimetable or getPlannedTimetable, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getCurrentTimetableA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that this is a read operation (retrieves data) and specifies what data is included (arrival/departure times, platform assignments, delays, real-time info). However, it doesn't mention potential limitations like rate limits, authentication needs, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently convey the tool's purpose and scope. The first sentence states the core function, and the second elaborates on what data is included. No wasted words, though it could be slightly more front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no annotations and no output schema, the description provides adequate context about what data is returned. However, it doesn't describe the return format, potential pagination, or error handling, which would be helpful given the lack of structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the evaNo parameter well-documented in the schema. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Ruft die aktuellen Fahrplandaten ab' - retrieves current timetable data) and resource ('einer bestimmten Bahnhofsstation' - of a specific railway station). It distinguishes from siblings by specifying it's for current/real-time data (vs. planned timetable or station search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool - for current timetable data with real-time information for the current operating day. It doesn't explicitly state when NOT to use it or name alternatives, but the context strongly implies this is for real-time vs. planned data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.