Server Quality Checklist

Tool Scores

• find_nearbyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, covering the safety profile. The description adds that it returns up to 'limit' elements with tags and a centre coordinate, giving a basic idea of the response shape. It lacks details on ordering, distance fields, or other response metadata, which is a moderate gap given no output schema is present. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with zero waste. The first sentence states the core function, the second gives the return shape, and the third provides a usage routing hint. It is front-loaded with the essential information and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and a nested tags object, the description provides the essential purpose, return shape, and a usage hint. It is mostly complete for calling the tool, though the phrase 'centre coordinate' is slightly ambiguous and it does not mention result ordering. Overall, it is adequate for a simple proximity query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents all five parameters including their types, ranges, and defaults. The description mentions tag filters and the limit but does not add new syntax or format details beyond what the schema already provides. Baseline 3 is therefore appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Find', the resource 'OpenStreetMap features', and the constraints 'matching tag filters within a radius of a point'. It also mentions the return of up to 'limit' elements with tags and a centre coordinate. While it positions itself as a good first tool for proximity queries, it does not explicitly differentiate from sibling tools like overpass_query or features_in_bbox, so it falls slightly short of the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit routing to count_features for larger or unknown result sets, which is a clear when-not and alternative. It also gives a concrete example use case ('what cafes are near X'). However, it does not discuss when to choose overpass_query or features_in_bbox, so the guidance is partial rather than comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• features_in_bboxA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover safety via readOnlyHint and openWorldHint. The description adds useful behavioral context by stating it returns up to 'limit' elements with tags and a centre coordinate, and it warns about performance with large boxes. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with no filler. The first sentence states the core operation, the second describes the return value, and the third provides necessary operational guidance. Every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description still explains the result shape (elements with tags and a centre coordinate) and the limit behavior. It also includes a practical scale warning. It lacks details like error behavior or exact response format, but an agent has enough to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description must help compensate. It adds the coordinate order and units ('south, west, north, east in decimal degrees') for the four bbox parameters, which is important semantic information not present in their schema entries. The tags and limit parameters are already well described in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action: find OpenStreetMap features matching tag filters within a bounding box, and describes the output shape. It is specific about the resource and inputs, but it does not explicitly distinguish itself from siblings like overpass_query or find_nearby, beyond one reference to count_features.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete operational guidance: keep boxes small and call count_features first for city-sized areas. This clearly states when this tool is appropriate versus count_features, though it does not explain when overpass_query or find_nearby would be better alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• count_featuresA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds useful behavior beyond the readOnlyHint: it is 'Cheap' and intentionally does not return features. This helps the agent understand cost and results without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, each earning its place: purpose first, then cost/use-case, then input options. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple count operation, the description covers the key invocation decisions and area semantics. Since there is no output schema, it could mention the exact return type, but the count semantics are clear enough from the name and description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25%, but the description compensates by explaining the two complementary input forms: point (lat, lon, radius_m) versus bounding box (south, west, north, east). This adds grouping and mutual-exclusivity semantics not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('Count'), resource ('OpenStreetMap features'), and scope ('matching tag filters in an area'). Adding 'without returning them' clearly distinguishes it from sibling tools that fetch features.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use this before pulling data when the result size is unknown.' This gives a concrete adoption context, though it does not explicitly name sibling alternatives or state when not to use it beyond what is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usageA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already signals no mutation, and the description adds the valuable detail that the call does not consume query quota. This is meaningful behavioral context beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, every clause earns its place. The main purpose is front-loaded, and the critical non-consumption fact plus usage guidance follow without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool, the description fully covers what the agent needs: what it returns, that it has no quota cost, and when to invoke it. No output schema is required to make this callable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so there is nothing for the description to document. The description instead clarifies what information the tool reports, which covers the semantic gap completely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific resource ('this API key') and exactly what gets checked: tier, limits, month-to-date quota, and request history. This clearly distinguishes it from sibling query tools like overpass_query or find_nearby.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use it to pace yourself when running many queries. It doesn't state exclusions or alternatives, but the use case is clear and no sibling competes for this role.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• overpass_queryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint/openWorldHint annotations, it discloses that data is full-planet OSM 'minutes behind live,' that queries default to 25 seconds without [timeout:], and where to learn syntax and server differences. These details materially shape safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four dense sentences carry high-value information with no filler. The most identifying action is front-loaded, followed by routing guidance and then concrete query construction rules.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, read-only semantics, and no output schema, the description covers what an agent needs to call the tool successfully: how to format the query, what the timeout behavior is, and pointers to authoritative references for syntax and server-specific differences.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single query parameter is already fully described in the schema with an example. The description adds extra meaning by emphasizing output caps, area bounding, and the [out:json] convention, which helps the agent construct valid queries rather than just knowing the parameter type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run a raw Overpass QL query against the Overspan hosted Overpass API.' It also distinguishes itself from the sibling helpers by saying to use it when 'the helper tools are too narrow,' making the tool's role clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance ('Use this when the helper tools are too narrow') and concrete procedural requirements: bound the query to an area, cap output, start with [out:json], and be aware of the 25-second timeout default. This is unusually actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.