Server Quality Checklist

Tool Scores

• get_ovfietsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool's function but does not add any behavioral context beyond that—such as whether it's read-only, requires authentication, has rate limits, or what the output format might be. This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's purpose without any unnecessary words or fluff. It is front-loaded and efficiently communicates the core function, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that likely returns availability data. It does not address what information is returned (e.g., number of bikes, timestamps), potential errors, or how to interpret results, leaving the agent with insufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'stationCode' well-documented in the schema itself. The description does not add any additional meaning or clarification beyond what the schema provides, such as examples of station codes or usage notes, so it meets the baseline for high schema coverage without enhancing parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('OV-fiets availability at a train station'), making it immediately understandable. However, it does not explicitly differentiate from sibling tools like 'get_station_info', which might also provide station-related data, leaving room for ambiguity in sibling context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lacks any mention of prerequisites, exclusions, or comparisons to sibling tools such as 'get_station_info' or 'get_arrivals', which could also be relevant for station-related queries, leaving the agent without contextual usage cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pricesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'Returns detailed pricing information with conditions and validity,' which gives some indication of output behavior. However, it doesn't address important behavioral aspects like whether this is a read-only operation, potential rate limits, authentication requirements, error conditions, or what happens with invalid inputs. For a tool with 10 parameters and no annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two clear sentences. The first sentence establishes the core purpose and scope, while the second describes the return value. There's no unnecessary verbiage or repetition. However, it could be slightly more front-loaded by mentioning the key parameters (origin/destination) that are required.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (10 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and return value but lacks important context about behavioral characteristics, error handling, and usage guidelines. The absence of an output schema means the description should ideally provide more detail about the return format, but it only gives a high-level overview ('detailed pricing information with conditions and validity').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are well-documented in the schema itself. The description adds minimal value beyond what's already in the schema - it mentions 'different travel classes, ticket types, and discounts' which corresponds to some parameters but doesn't provide additional context about parameter interactions or usage patterns. With complete schema documentation, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get price information for domestic train journeys' with specific details about what's included (travel classes, ticket types, discounts). It distinguishes itself from siblings like get_arrivals or get_departures by focusing on pricing rather than schedule information. However, it doesn't explicitly differentiate from potential pricing-related siblings that might exist in other contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While it mentions what the tool does, it doesn't indicate scenarios where this tool is appropriate versus other tools like get_travel_advice (which might also include pricing) or when not to use it. There's no mention of prerequisites, limitations, or comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_station_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, it doesn't specify whether this requires authentication, has rate limits, returns structured data, or handles errors. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a straightforward lookup tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate but incomplete. It covers the basic purpose but lacks behavioral context, usage differentiation, and output information that would help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information beyond what's already in the schema (which has 100% coverage). It doesn't explain how the 'query' parameter works with station names versus codes, or provide examples. With complete schema documentation, the baseline is 3 as the description doesn't enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'detailed information about a train station', making the purpose immediately understandable. However, it doesn't differentiate this tool from potential sibling tools that might also retrieve station information in different contexts (like get_arrivals or get_departures), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like get_arrivals and get_departures that also relate to stations, there's no indication whether this tool should be used for static station metadata versus dynamic schedule data, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_departuresB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'real-time' information and 'returns a list of upcoming departures' which provides basic behavioral context, but doesn't address important aspects like rate limits, authentication requirements, data freshness guarantees, error conditions, or pagination behavior. The description is insufficient for a tool with 5 parameters and no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded with the core purpose. The first sentence clearly states what the tool does, and the second explains the return format. There's minimal redundancy, though the second sentence could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description provides adequate basic context about purpose and return format but lacks important behavioral details. The description covers the 'what' but not the 'how' or constraints. Given the complexity and lack of structured metadata, it should provide more operational guidance to be truly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all 5 parameters. The description doesn't add any parameter-specific information beyond what's in the schema - it doesn't clarify station code formats beyond 'NS Station code', explain UIC code usage, or provide examples for the dateTime parameter. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get real-time departure information'), resource ('trains from a specific station'), and scope ('including platform numbers, delays, route details, and any relevant travel notes'). It distinguishes from sibling tools like get_arrivals (departures vs arrivals) and get_station_info (departure info vs general station info).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for train departure information, but doesn't explicitly state when to use this tool versus alternatives like get_arrivals for arrivals or get_travel_advice for route planning. No explicit exclusions or prerequisites are mentioned, leaving usage context somewhat implied rather than clearly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_arrivalsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the core functionality (real-time arrival information) and output format (list of upcoming arrivals with timing, origin, and status). However, it doesn't mention important behavioral aspects like rate limits, authentication requirements, data freshness guarantees, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two well-structured sentences. The first sentence establishes purpose and scope, while the second clarifies the return format. Every word serves a clear purpose with zero redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only query tool with 5 parameters and no output schema, the description provides adequate but minimal context. It covers the core functionality and return format, but lacks details about error handling, data limitations, or practical usage patterns that would help an agent use it effectively in complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions the general data returned but doesn't provide additional context about parameter interactions or usage patterns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get real-time arrival information'), resource ('trains at a specific station'), and scope ('upcoming arrivals'). It distinguishes from siblings like get_departures by focusing on arrivals rather than departures, and from get_station_info by providing dynamic arrival data rather than static station details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking train arrivals at stations, but doesn't explicitly state when to use this tool versus alternatives like get_departures or get_travel_advice. No specific exclusions or prerequisites are mentioned, leaving the agent to infer appropriate contexts from the tool's purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_travel_adviceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors like returning multiple route options with status and crowding information, but it lacks details on error handling, rate limits, or authentication needs, which are important for a tool with real-time data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently lists key features in two sentences without redundancy. Every sentence adds value, such as specifying optimization options and return data, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a route planning tool with real-time data and no output schema or annotations, the description is adequate but incomplete. It covers the basic functionality and return types, but lacks details on output structure, error cases, or limitations, which could hinder an agent's ability to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds marginal value by implying the tool handles both departure and arrival times, but it does not provide additional syntax or format details beyond what the schema specifies, such as examples for dateTime format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get detailed travel routes') and resources ('between two train stations'), distinguishing it from siblings like get_arrivals or get_departures by focusing on route planning rather than station-specific information. It specifies the scope includes transfers, real-time updates, and journey duration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('plan trips for immediate departure or for a specific future time') and mentions optimization options, but it does not explicitly state when not to use it or name alternatives among sibling tools, such as using get_prices for fare information instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_disruptionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses behavioral traits such as the types of information returned (maintenance work, unexpected disruptions, alternative transport options, etc.) and filtering capabilities, but lacks details on rate limits, authentication needs, or error handling. It adequately describes the tool's function without contradicting any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the first sentence stating the core purpose and subsequent sentences adding useful details about return values and filtering options. Every sentence earns its place without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is fairly complete. It explains what the tool does, what information it returns, and filtering options. However, it could be more complete by detailing output structure or error cases, though this is mitigated by the straightforward nature of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (isActive and type) fully. The description adds minimal value beyond the schema by mentioning filtering for 'active disruptions' and 'specific disruption types', but does not provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('comprehensive information about current and planned disruptions on the Dutch railway network'), and distinguishes it from siblings like get_arrivals, get_departures, or get_travel_advice by focusing specifically on disruption information rather than schedules, pricing, or general advice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Get comprehensive information about current and planned disruptions'), but does not explicitly state when not to use it or name specific alternatives among the sibling tools. It implies usage for disruption-related queries without excluding other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_time_in_rfc3339A
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the timezone behavior ('Europe/Amsterdam timezone') and output format ('RFC3339 format'), but does not mention potential behavioral traits like rate limits, authentication needs, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the core purpose, and the second provides usage context. Every sentence earns its place, and it is appropriately front-loaded with the main functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is largely complete. It explains what the tool returns (time in RFC3339 format) and its usage context. A minor gap is the lack of explicit mention of the return value structure, but this is acceptable for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage. The description appropriately does not discuss parameters, and the baseline score of 4 reflects that no parameter documentation is needed for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the current server time') with precise resource details ('Europe/Amsterdam timezone' and 'RFC3339 format'). It distinguishes itself from sibling tools (e.g., get_arrivals, get_departures) by focusing on time retrieval rather than transportation data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('as input for other tools that require date-time parameters'), but does not explicitly state when not to use it or name specific alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.