Server Quality Checklist

Tool Scores

• route-overviewB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the output format ('JSON') but lacks critical details: whether this is a read-only operation, if it requires authentication, rate limits, error conditions, or what happens with invalid locations. For a routing tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states purpose and key outputs, the second specifies the result format. It's front-loaded with essential information and has no wasted words, though it could be slightly more comprehensive given the lack of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a routing tool with 3 required parameters, no annotations, and no output schema, the description is minimally adequate. It covers basic purpose and output format but lacks behavioral context, parameter guidance, and error handling. The absence of output schema means the description should ideally explain return values more thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (2 of 3 parameters have descriptions). The description adds minimal value beyond the schema—it implies 'locations' parameter usage but doesn't explain parameter interactions or provide examples. With moderate schema coverage, the baseline of 3 is appropriate as the description doesn't significantly compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get high-level routing information'), resource ('between two or more locations'), and output details ('travel time, distance, and an encoded polyline of the route'). It distinguishes itself from siblings like geocode or isochrone by focusing on routing between multiple points rather than address conversion or area-based calculations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like isochrone (for travel-time areas) or static-map (for visual route display), nor does it specify prerequisites or exclusions. Usage context is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• static-mapB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool generates a PNG image but doesn't mention output format details, size limitations, rate limits, authentication needs, or error conditions. The description is minimal and lacks crucial behavioral context for a tool that produces visual output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Generate a PNG map image') and includes essential optional features. Every word earns its place with zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the PNG output contains, how large images can be, coordinate system details, or error handling. The description leaves too many behavioral aspects undefined given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds marginal value by mentioning 'markers and a line' which corresponds to the 'markers' and 'encodedPolyline' parameters, but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a PNG map image of an area' with specific optional features ('including markers and a line'). It distinguishes from siblings like 'geocode' (address conversion) and 'route-overview' (likely textual routing) by focusing on visual map generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through examples ('e.g. to draw a route or a boundary'), but doesn't explicitly state when to use this versus alternatives like 'route-overview' or 'isochrone'. It mentions that either markers or a polyline must be specified, which provides some contextual guidance but lacks explicit sibling differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• isochroneA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'Returns GeoJSON polygons representing the reachable areas,' which covers the output format. However, it lacks critical behavioral details: whether this is a read-only operation, computational cost, rate limits, authentication requirements, or what happens with invalid inputs. For a geospatial computation tool with no annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first defines the tool's purpose and scope, the second specifies the return format. It's front-loaded with the core functionality and contains zero redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (geospatial computation with multiple parameters and nested objects) and 100% schema coverage but no annotations or output schema, the description is minimally adequate. It covers the basic purpose and output format but lacks behavioral context that would be crucial for an AI agent to use this tool effectively in production scenarios. The absence of output schema means the description should ideally explain return values more thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'time or distance constraints' and 'single location,' which are already clear from parameter names and descriptions. The baseline score of 3 is appropriate when the schema does the heavy lifting, though the description doesn't provide additional context like parameter interactions or constraints beyond what's in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Generate isochrone contours') and resources ('areas reachable within specified time or distance constraints from a single location'). It distinguishes itself from siblings like geocode, route-overview, and static-map by focusing on reachability analysis rather than address lookup, routing, or map generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'areas reachable within specified time or distance constraints,' suggesting it's for accessibility analysis. However, it provides no explicit guidance on when to use this tool versus alternatives like route-overview (which might provide point-to-point routing) or when not to use it. No prerequisites or sibling tool comparisons are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• time-and-zone-infoA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the output format (UTC offset, special offset, IANA TZID, RFC 28222 timestamp) but doesn't mention error handling, rate limits, authentication needs, or whether this is a read-only operation. It adds some context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose and output details without unnecessary words. It's front-loaded with the core function and provides essential information concisely, with every element earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is reasonably complete. It explains what the tool does and what it returns, but could benefit from more behavioral context like error cases or usage constraints. Without an output schema, the description adequately covers the return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for both lat and lon parameters. The description doesn't add any parameter-specific semantics beyond what the schema already provides, such as explaining coordinate systems or precision. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the current time and zone info') and the resource ('at any point (geographic coordinates)'), distinguishing it from sibling tools like geocode or route-overview which handle different geographic functions. It precisely defines what the tool does without being vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like bulk-geocode or isochrone, nor does it mention any prerequisites or exclusions. While the purpose is clear, there's no explicit context for usage relative to sibling tools, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• geocodeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (coordinates, bounding box, local context, and additional info like Wikipedia ID) and mentions that information is 'subject to availability,' which adds useful context about potential variability. However, it doesn't cover important behavioral aspects like rate limits, authentication needs, error conditions, or whether it's a read-only operation (though 'look up' implies non-destructive).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose and return values, and the second sentence adds supplementary information about additional data. Every sentence earns its place with no wasted words, making it efficient and easy to understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, nested objects, no output schema), the description is reasonably complete. It explains what the tool does and what information it returns, which is adequate for a geocoding function. However, without annotations or an output schema, it could benefit from more detail about the structure of returned data or error handling, though the high schema coverage helps compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, which is high, so the baseline is 3 even though the description doesn't add parameter-specific information. The description mentions the types of queries (street address, POI, area) which aligns with the 'query' parameter, but doesn't provide additional semantic context beyond what's already documented in the detailed schema descriptions for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('look up') and resources ('street address, POI, or area'), and distinguishes it from siblings by focusing on single-location geocoding (vs. bulk-geocode for multiple locations). It explicitly lists the types of information returned, making the function unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (for looking up addresses, POIs, or areas to get geographic information), but doesn't explicitly mention when not to use it or name alternatives. It implies usage for single queries rather than bulk operations, which differentiates it from 'bulk-geocode', but lacks explicit exclusions or comparisons to other siblings like 'isochrone' or 'route-overview'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bulk-geocodeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it's a batch operation, returns only first result per query, includes what geographic information is returned (coordinates, bounding box, local context), and warns about limitations with POIs. It doesn't mention rate limits, authentication needs, or error handling, which keeps it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with three sentences that each earn their place: first states the core functionality, second provides critical usage guidance, third details return values. No wasted words, and the most important information (bulk geocoding for addresses only) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description does well by explaining the batch nature, return format, and content of results. It could be more complete by mentioning error handling for invalid addresses or whether all items must succeed for the batch to complete, but it covers the essential context given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description doesn't add any parameter-specific information beyond what's already documented in the schema. It focuses on overall tool behavior rather than explaining individual parameters like 'items', 'countryFilter', or 'layer'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('perform multiple address geocoding operations in a single request') and distinguishes it from the sibling 'geocode' tool by emphasizing bulk processing. It explicitly identifies the resource being operated on (addresses) and the output format (JSON list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('only use this for addresses') versus when not to ('strongly discouraged' for POIs), and explains why (many places with same name exist). It also implies an alternative approach for POIs (using general knowledge or multiple queries), though it doesn't name specific sibling tools as alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.