Server Quality Checklist

Tool Scores

• find_nearest_stationsC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation contradiction: description says 'find' (non-destructive) but annotations have destructiveHint=true, implying potential data modification. No additional behavioral context provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no wasted words, front-loads the essential action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description should hint at return type or behavior. Fails to mention destructive nature suggested by annotations, leaving agent uncertain about side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. Description adds no extra meaning beyond schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the purpose (find fuel stations nearest to coordinate), but doesn't explicitly differentiate from sibling tool 'get_stations_in_area' which likely also returns stations by area.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives like get_stations_in_area or find_route_stations. Context signals show sibling tools but description offers no usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_routeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not address the destructiveHint annotation, which suggests side effects. 'Calculate' implies a read-only computation, but the tool may modify state. No explanation of what gets destroyed or other behavioral traits is given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no redundant words. It is front-loaded with the core purpose, though it could be expanded to include output hints without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks information about the return value (e.g., does it return geometry, duration, distance?) and does not clarify the destructive hint. Given no output schema, these omissions reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are described in the input schema with clear names and types. The description adds no extra meaning beyond 'driving route', so it meets the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Calculate a driving route between two points', specifying the action and resource. It distinguishes from sibling tools like geocode or find_nearest_stations, which are about location data and stations rather than route calculation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For instance, it could mention that for station-specific queries, sibling tools like find_route_stations are appropriate, but it does not.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_route_stationsC
  2.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description claims a read operation ('Find') while annotations set destructiveHint=true, suggesting data modification. This contradiction is not addressed. No additional behavioral context beyond the purpose is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no wasted words. However, it is too brief given the contradiction and lack of usage guidance, slightly reducing effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite low complexity, the description is incomplete: it does not explain the output, does not resolve the annotation contradiction, and assumes the agent knows what 'route corridor' means. A more complete description would include output format and usage notes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with descriptions for each parameter (corridor_km, fuel, geometry). The tool description does not add new semantic details beyond what the schema provides, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Find fuel stations along a route corridor' clearly states the verb and resource, and implies the corridor context. It distinguishes from siblings like 'find_nearest_stations' (point-based) and 'get_stations_in_area' (area-based), though not explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives like find_nearest_stations or get_stations_in_area. Usage is implied by the description and the required geometry parameter (GeoJSON LineString), but no exclusion or prerequisite details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stations_in_areaB
  3.1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description says 'Get' implying a read-only operation, but annotations indicate readOnlyHint=false and destructiveHint=true, suggesting potential side effects or data destruction. This is a direct contradiction, and the description fails to disclose any behavioral traits beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no wasted words. However, it could be slightly more informative without losing conciseness, e.g., by noting the return type or limitations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the potential for destructive behavior (per annotations), the lack of explanation about side effects, return format, or operational limits makes the description incomplete. The minimal information does not compensate for the complexity implied by annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters (bbox format and fuel type codes). The description adds no extra meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get fuel stations within a bounding box' clearly states the verb (Get), resource (fuel stations), and scope (bounding box). It effectively differentiates from sibling tools like find_nearest_stations (point-based) and find_route_stations (route-based).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a bounding box is available, but it does not explicitly state when to use this tool over alternatives like find_nearest_stations or calculate_route. No guidance on prerequisites or limitations is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• geocodeB
  3.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description says 'search' (read-like), but annotations set destructiveHint=true, suggesting modification. This contradiction confuses agent behavior. Description adds no further behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence is concise and front-loaded, but lacks structure (e.g., no sections) and provides minimal detail for a 3-param tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With contradictory annotations and no output schema, description misses essential return format info and fails to disclose behavioral implications, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, but description adds geocoding context ('by name') beyond param details. This clarifies the tool's purpose and differentiates forward vs reverse geocoding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'search' and resource 'location by name' (geocoding), distinguishing it from sibling tools like calculate_route and find_nearest_stations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies geocoding use via 'search by name' but provides no explicit when-to-use vs alternatives, no exclusions, and no context-sensitive guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.