Server Quality Checklist

Tool Scores

• spatial_joinA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes core behavior (returns properties of containing polygon) and efficiency, but omits details on output format, handling of points outside polygons, or error conditions. With no annotations, the description carries full burden but doesn't cover edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose comparison to sibling prerequisite. No wasted words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers main usage and prerequisite, but lacks output details and error cases. Given no output schema, description could be slightly more complete, but is adequate for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented. Description adds value by reiterating prerequisite (dataset id from list_datasets) and coordinate requirement, but does not significantly expand beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it enriches coordinates with polygon attributes and compares to sibling check_location, making the purpose and distinction obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear prerequisite (call list_datasets first) and notes coordinate requirements. Implicitly suggests when to use this over check_location (multiple points), but lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_distanceA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It discloses the WGS84 ellipsoid model and return format (meters and miles). However, it omits error handling, input validation, or precision limits. Adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, and provides essential details without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains return values (meters and miles). Parameters are fully described in schema. Sibling tools are clearly distinct, and context is complete for this simple calculation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The tool description does not add significant meaning beyond the schema's parameter descriptions (e.g., does not specify valid coordinate ranges or decimal degree format).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it calculates geodesic distance between two points on Earth's surface. It specifies high accuracy using WGS84 ellipsoid, and the purpose is distinct from sibling tools (check_location, enrich_location, find_nearest, list_datasets, spatial_join).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'No dataset needed', implying standalone use, but lacks explicit guidance on when to use this tool versus alternatives (e.g., spatial_join). There are no exclusions or when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_datasetsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return content (public and custom datasets) and important ID usage, but lacks details on pagination, ordering, or read-only nature, especially since no annotations exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences efficiently convey purpose, content, and a critical usage instruction with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple listing tool, but missing details about response structure (e.g., additional dataset properties) given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and 100% schema coverage, the description adds value by explaining what datasets are returned and emphasizing ID usage, meeting the baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all geospatial datasets, distinguishing it from sibling tools that perform operations on locations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implicitly recommends using this tool first by stating to always use the dataset id in other tools, but does not explicitly say when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_locationA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return behavior ('Returns properties of all matching features, or indicates no match') but does not disclose potential side effects or permission requirements. Since no annotations are provided, the description carries the full burden, and it falls short of fully disclosing operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three efficient sentences: first defines purpose, second describes output, third states requirements and disclaimers. Front-loaded with the core action, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, it adequately describes the return value. For a simple point-in-polygon query with three parameters, it covers inputs, prerequisites, and what to expect. Minor gap: no mention of error handling or output data format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal extra meaning beyond the schema, only noting that the dataset id comes from list_datasets. No additional constraints or examples beyond what is in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource ('check which feature in a polygon dataset contains a given coordinate') and explicitly distinguishes from siblings by stating it is NOT a geocoder and requires numeric coordinates, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states prerequisites (call list_datasets first to get the dataset id) and provides a clear when-not-to-use condition (not for addresses or place names), guiding appropriate usage against alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• enrich_locationA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description discloses key behaviors: queries all datasets in one call, returns what matched, optional radius. Missing details on error handling or limits, but sufficient for typical use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each essential: purpose, usage note, and critical exclusion (geocoder). Front-loaded, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all essential aspects for a simple lookup tool with no output schema. Explains input, behavior, and what it returns. Could mention limits like max radius, but schema already provides that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds minor context ('simplest way') but does not significantly enhance understanding beyond schema. No new constraints or formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb ('Get location attributes'), specific resource ('location attributes'), and distinguishes from siblings by stating it's not a geocoder and doesn't need dataset id. Defines scope exactly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('simplest way') and when not ('NOT a geocoder'). Provides context on optional radius. Does not explicitly name alternative sibling tools but implies differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_nearestA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses behavior: returns features sorted by distance (meters and miles), measured to feature boundary, and includes reverse lookup (containing feature with distance=0). No side effects or auth needs mentioned, but adequate for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no wasted words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description explains return format (features sorted by distance, in meters and miles). Also mentions reverse lookup behavior. Almost complete; could mention return structure but sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage 100%, and the description adds context: explains that dataset UUID comes from list_datasets, and mentions max values for n (20) and radius (500000) which are already in schema. Provides meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool finds nearest features (counties, districts, ZIP codes) to a coordinate, with a specific verb 'find' and resource 'nearest features'. Distinguishes from siblings by mentioning proximity queries and reverse lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says use for proximity queries, finding what's nearby, ranking by distance, and reverse lookup. Does not explicitly state when not to use or compare to sibling tools like calculate_distance, but the description provides good context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.