Server Quality Checklist

Tool Scores

• fuel_nearby_stationsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds ordering behavior (nearest first) and field scope. However, it doesn't disclose details like pagination limits behavior or what happens when max_results/radius are exceeded, though the schema constraints (min/max) partially cover this. With strong annotation coverage, the description's additions are sufficient but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single concise sentence that efficiently enumerates what's returned (prices, brand, amenities, flags) plus ordering (nearest first). Zero waste, front-loaded with the action. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a good output schema present, an output_format annotation available, and strong safety annotations (readOnly, idempotent, non-destructive), the description doesn't need to explain return values. It covers the core result content and ordering. For a list-type read tool with output schema available, this is reasonably complete, though slightly more sibling differentiation would push it to 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% — only the location parameter has a description in the schema ('UK postcode, outcode or lat,lon'). The description itself doesn't elaborate on max_results or radius_miles semantics, though the schema provides min/max/default values for those. Since location is the only required param and is well-described in the schema, the description adds minimal value but the critical param is covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool lists forecourts near a location (nearest first) with specific attributes included: fuel prices, brand, amenities, and motorway/supermarket flags. It also notes ordering (nearest first). This distinguishes it from siblings like fuel_find_cheapest (which focuses on cheapest) and fuel_area_summary (area aggregation). Specific verb+resource+scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it's for finding nearby stations with full detail, but doesn't explicitly state when to use it vs fuel_find_cheapest or fuel_area_summary. There's no when-not guidance or named alternatives. The 'nearest first' ordering and comprehensive field listing give some usage context, but explicit differentiation from siblings is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fuel_find_cheapestA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is well-covered. The description adds useful context: official Fuel Finder data source and 30-minute freshness window. However, it doesn't disclose behavior about edge cases (no stations found, unknown fuel type, invalid postcode) or what the response structure looks like. With strong annotations, this is acceptable but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact single sentence that packs in purpose, ranking behavior, median comparison, unit, data source, and freshness into an efficient package. It's front-loaded with the core function ('Find the cheapest fuel') immediately. Slightly dense but well-organized; the information-to-word-count ratio is excellent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the return-value format is handled structurally. The description covers data source, freshness, pricing unit, ranking, and median comparison — all the key semantics. Given 4 params, 50% schema coverage, rich annotations, and an output schema, the description is reasonably complete. Could mention edge cases or failure behavior, but overall it's adequate for reliable tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: 'location' and 'fuel' have descriptions, but 'top_k' and 'radius_miles' do not. The description clarifies the pricing unit (pence per litre) and data provenance, which adds value. However, the top_k (default 10, max 25) and radius_miles bounds are only discoverable from the schema, not reinforced in the description. Baseline 3 is fair since half the params are documented in schema and the description adds the freshness/data-source context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Find the cheapest fuel near a location, ranked by price', with each station's price compared to the local median. The verb 'find' + resource 'cheapest fuel' + clear ranking criterion distinguishes this from siblings like fuel_nearby_stations (which likely lists stations without the cheapest-ranking emphasis) and fuel_area_summary (which summarizes an area).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for cost-conscious fuel searches by ranking by price and comparing to local median. It doesn't explicitly name alternatives or exclusion criteria, but the 'ranked by price' framing and the sibling context (fuel_nearby_stations for general listing, fuel_data_status for data freshness) gives reasonable usage guidance. No explicit when-not-to-use, but the purpose is clear enough to differentiate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fuel_area_summaryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, openWorldHint=true, and destructiveHint=false, so the safety profile is covered by metadata. The description adds value by disclosing the aggregation behavior (statistics per fuel type) and the analytical intent (judging price competitiveness), which goes beyond the structured metadata. It doesn't discuss potential rate limits, data freshness/coverage limitations, or what happens for areas with no stations, but it doesn't contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero filler. The first sentence states the core functionality with specific outputs (min/median/max, cheapest station), and the second adds practical usage guidance. Every sentence earns its place and the description is appropriately front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description doesn't need to explain return values. The description covers the aggregation purpose, the location parameter context, and the analytical use case. Given it's a relatively simple two-parameter read-only aggregation tool with good annotations and output schema, the description is essentially complete; minor gaps around data coverage limitations or radius semantics prevent a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: the 'location' param is described ('UK postcode, outcode or lat,lon') but 'radius_miles' has no description. The tool description adds some context by indicating the area scope concept, but neither the description nor the schema clarifies radius_miles semantics beyond its default (10) and range. With partial schema coverage, the description partially compensates but doesn't fully cover the undocumented parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource+scope: 'Per-fuel price statistics for an area: min/median/max and the cheapest station for each fuel.' It specifies exactly what outputs (statistics and cheapest station) and clearly distinguishes itself from siblings like fuel_find_cheapest (which finds a single cheapest) and fuel_nearby_stations (which lists stations). It even frames its purpose relative to a user need ('judge whether a price you've seen is actually good locally').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context ('Use it to judge whether a price you've seen is actually good locally'), which effectively communicates when to use this aggregation tool versus alternatives. It doesn't explicitly state when NOT to use it or name sibling alternatives, but the purpose statement strongly implies the appropriate use case relative to the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fuel_data_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is fully covered by structured data. The description adds the diagnostic purpose (checking credentials, stations, cache age to diagnose empty results), which is useful context beyond what annotations provide. However, it doesn't describe the output format or how the agent should interpret the return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The description is front-loaded with a clear verb ('Check') and immediately follows with the diagnostic use case. Both sentences earn their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter diagnostic tool with a full safety profile from annotations and an output schema, the description is reasonably complete. It states what's checked (credentials, stations, cache age) and when to use it (diagnose empty results). The tool is simple enough that this level of detail is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and 100% schema description coverage (trivially, since there are no properties). The baseline of 4 for zero params applies here. The description doesn't need to document parameters that don't exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') with clear resources (server's data: credentials, stations, cache age) and explicitly states its diagnostic purpose. It clearly distinguishes from sibling tools, which all perform lookups or queries, whereas this tool inspects the data layer itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly gives a usage context: 'Use this to diagnose empty results.' This tells the agent when to invoke this tool versus the sibling lookup tools. However, it doesn't provide exclusions or when-not-to-use guidance, which is a minor gap for a diagnostic tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.