Server Quality Checklist

Tool Scores

• get_route_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the tool returns 'detailed route information' but does not state key behavioral traits such as read-only nature, potential latency from external API calls, authentication requirements, or any constraints. This is insufficient for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with clear Args and Returns sections, no redundant sentences. Every part contributes to understanding. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking annotations, the description covers the main purpose, parameters, and return content fairly well. The presence of an output schema reduces the need to detail return structure. Some gaps remain (e.g., slug constraints), but overall it is adequate for a simple two-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the input schema has 0% description coverage, the description adds meaning by specifying that from_city and to_city are origin and destination city slugs with examples (e.g., 'rome', 'milan'). However, it does not clarify slug format, case sensitivity, or provide an exhaustive list, so the added value is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed route information between two Italian cities, including transport modes, price ranges, and travel times. This verb+resource combination is distinct from siblings like get_cheapest_travel_day (which focuses on cheapest day) and list_italy_routes (which likely lists routes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus its siblings. The description does not mention alternative tools or conditions for exclusion, leaving the agent to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_italy_routesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only mentions the return format (JSON list with cheapest prices and number of offers) but does not disclose behavioral traits such as read-only nature, performance considerations, or lack of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences) and front-loaded. It directly states the action and return value without any filler, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no input parameters and an output schema exists, the description adequately covers the tool's purpose and return structure. It could be improved by noting any implicit limitations (e.g., geographic scope), but it is sufficient for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter documentation is needed. The description adds value by specifying the output structure (cheapest prices, number of offers), though this pertains to output rather than parameters. Baseline score of 4 for zero parameters is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all available transport routes between Italian cities', which is a specific verb and resource. It distinguishes from siblings like get_cheapest_travel_day and get_route_info by implying a broad, unfiltered listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings like search_italy_transport or get_route_info. The description does not mention any context or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_italy_transportA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It states the return type (JSON with options, prices, operators) but does not disclose any side effects, rate limits, authentication needs, or pagination. For a search tool, the description is adequate but lacking in detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: one line for purpose, then args, then returns. No redundant information. It is front-loaded with the search purpose and structured clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (though not fully shown), the description adequately covers purpose, parameters, and return type. It mentions Italian cities but could specify transport modes (train, bus?) or add more examples. Overall, it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by listing all three parameters with explanations and examples (e.g., 'from_city: Origin city slug (e.g. ''rome'', ''milan'', ...)'). The date parameter is described as optional with format YYYY-MM-DD. This adds meaning beyond the schema's bare titles and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for transport options between two Italian cities.' It specifies the verb 'search' and the resource 'transport options'. It distinguishes itself from siblings like list_italy_routes and get_cheapest_travel_day by focusing on searching rather than listing or cheapest day.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, when not to use, or how it relates to siblings like get_cheapest_travel_day. The user must infer usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cheapest_travel_dayA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It states the return value (day-by-day price breakdown) but does not disclose side effects, whether it is read-only, or any authentication requirements. The behavior is minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a clear one-sentence purpose followed by labeled Args and Returns sections. Every part adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic functionality and return type, but given the existence of sibling tools related to routes and transport, it lacks context on prerequisites (e.g., whether the route must exist) or error conditions. The output schema exists but is not shown, so the description adequately explains returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description adds meaning to both parameters: 'Origin city slug' and 'Destination city slug' with examples (e.g., 'rome', 'milan', 'florence'). This helps the agent understand the expected format and provides value beyond the schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to find the cheapest day of the week for travel between two Italian cities. The verb 'Find' and specific resource 'cheapest day of the week to travel' are distinct from sibling tools like 'get_route_info' and 'list_italy_routes', which focus on routes and transport options respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through its purpose, but it does not explicitly state when to use this tool versus alternatives. No guidance on prerequisites, limitations (e.g., only Italian cities), or when to choose another tool is provided. The sibling tools exist but are not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.