Server Quality Checklist

Tool Scores

• search_tripsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by detailing what the tool returns (trip options with specific fields like duration, transfers, status, price), default behaviors (e.g., date_time defaults to current time), and practical usage notes (e.g., using search_stations for codes). It doesn't mention rate limits or authentication needs, but covers core behavior adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns, example) and front-loaded key information. It's appropriately sized but could be slightly more concise by integrating the example more tightly or trimming some redundancy in parameter explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, no annotations, but has output schema), the description is highly complete. It covers purpose, usage, all parameters with semantics, return structure, and examples. The output schema exists, so the description appropriately focuses on explaining the return values' meaning rather than just structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed semantics for all 8 parameters: examples (e.g., 'ut' for Utrecht), explanations (e.g., search_for_arrival controls date_time interpretation), default values, and valid options (e.g., travel_class values). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Search for train trips between two stations with pricing information' and identifies it as 'the main tool for route planning.' It distinguishes from sibling tools by specifying it returns trip options with connections, travel times, and prices, unlike get_departures (likely real-time departures) and search_stations (station lookup).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating this is 'the main tool for route planning' and includes guidance on using search_stations to find station codes for parameters. However, it doesn't explicitly state when to use this vs. get_departures (e.g., for planning vs. real-time info) or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_stationsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behaviors: it describes the return format (dictionary with stations list and count), default and max values for limit, and search constraints (minimum 2 characters for query). It doesn't cover rate limits or authentication needs, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by detailed parameter explanations and examples. Every sentence adds value—no redundancy or fluff—making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations), the description is complete: it explains purpose, usage, parameters, return values, and provides examples. With an output schema present, it doesn't need to detail return structure further, and it adequately covers all necessary context for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the schema: explains that 'query' searches station names with a 2-character minimum and can be empty to list all, clarifies 'country_codes' format and provides examples, and specifies 'limit' default and max values. This covers all parameters thoroughly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('search') and resource ('train stations') with filtering capabilities ('by name or filter by country'). It distinguishes from siblings like 'get_departures' (focused on departure times) and 'search_trips' (focused on trip planning) by emphasizing station code retrieval for trip planning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this tool to find station codes needed for trip planning,' providing clear context for when to use it. However, it doesn't mention when NOT to use it or explicitly compare to sibling tools like 'search_trips,' which might be a better alternative for certain trip-related queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_departuresA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (real-time updates about delays, cancellations, and platform changes) and provides detailed return structure. However, it doesn't mention potential limitations like rate limits, authentication requirements, or error conditions that might be important for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage, args, returns, examples) and every sentence adds value. It's appropriately sized for a tool with 3 parameters and detailed return structure, with no redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (real-time data with multiple parameters) and the presence of an output schema, the description provides complete context. It covers purpose, usage guidelines, parameter semantics, and return structure. The output schema handles the detailed return format, so the description doesn't need to duplicate that information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter documentation. It explains what each parameter means, provides examples (e.g., 'ut' for Utrecht Centraal), specifies defaults (10 for max_journeys, current time for date_time), and gives constraints (max: 40). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Get') and resource ('upcoming train departures for a specific station'). It distinguishes from sibling tools by focusing on departure information rather than station search or trip planning, making it easy for an agent to understand when this tool is appropriate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this to check the departure board at a station' and provides a clear alternative ('Use search_stations to find codes') for one of the parameters. It distinguishes from sibling tools by focusing on real-time departure information rather than station search or trip planning, giving the agent clear guidance on when to use this specific tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.