Server Quality Checklist

Tool Scores

• find_stationA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that it returns matching options as 'a formatted string showing matching stations with their IDs and full names,' and notes it's useful for checking station names. This provides a decent picture of the operation and output, though it doesn't explicitly mention non-destructive behavior or edge cases like no results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with the main purpose stated in the first sentence, followed by usage context and a structured 'Args' and 'Returns' section. Every sentence adds value with no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description covers purpose, usage context, parameter semantics, and return format. It provides enough information for an agent to select and invoke the tool correctly without confusion, making it complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only provides the parameter name and type, with 0% description coverage. The description compensates fully by including an 'Args' section that explains station_name with concrete examples ('Paris', 'Munich', 'Lyon'), making the parameter's meaning and expected format clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search for a train station by name and return matching options.' This uses a specific verb and resource, and it distinguishes itself from sibling tools like search_trains (which searches journeys) and get_train_prices (which gets prices).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says it is 'Useful for checking station names before searching for journeys, or when you're not sure of the exact station name,' which gives clear context for when to use it. It doesn't explicitly name alternatives, but the context implies it should be used before journey searches, which is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_trainsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses default behavior for missing departure_datetime (searches from current time), pagination default of 10 results per page, and the need to request different pages. It also explains flexible date formats, offering rich behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured as a docstring with a one-line summary followed by per-parameter explanations. Each entry is concise yet informative, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all four parameters, explains default behaviors, and provides usage context. Since an output schema exists, return values need not be described. This is complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain each parameter. It does so thoroughly: origin and destination include examples, departure_datetime lists multiple accepted formats and default behavior, and page explains the default and result count. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search for train journeys between two stations with pagination,' clearly identifying the action (search), resource (train journeys), and key behavior (pagination). This distinguishes it from sibling tools like find_station and get_train_prices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear context for when to use the tool—searching train journeys between two stations—but does not explicitly state when-not-to-use it or mention alternative tools. This matches 'clear context, no exclusions'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_train_pricesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it 'attempts to scrape prices' and may fail due to anti-scraping measures, API changes, or ToS restrictions. It also notes that returns are 'if available', setting accurate expectations. This is highly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a front-loaded warning, a list of alternatives, an Args section, and a Returns section. Every sentence adds value, and the format is clear despite being slightly verbose due to the necessary experimental warnings.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an experimental scraping tool, the description covers all necessary aspects: what it does, its limitations, all parameters, and the return format. It is self-contained and provides enough context for an agent to invoke it correctly, even without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description thoroughly explains every parameter with examples and defaults, compensating for 0% schema coverage. However, there is a minor inconsistency: departure_datetime is described as 'default: today' while the schema default is null, which could cause minor confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: 'Get train prices' with explicit scope ('from SNCF'). It distinguishes itself from sibling tools like 'search_trains' and 'find_station' by focusing on price retrieval, and adds context as an 'Educational Proof of Concept'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description is explicit about when to use it (educational/testing) and explicitly warns against production use, offering concrete alternatives (Lyko SNCF Connect API, Trainline API, official SNCF partnerships). This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.