Server Quality Checklist

Tool Scores

• get_datasetA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does disclose the returned content and the exact/partial matching behavior, which is useful. However, it does not mention how ambiguous partial matches are handled (e.g., multiple matches, error vs. arbitrary selection), nor any potential side effects or permissions, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action, scope, and key return details without redundancy or filler. Every clause contributes useful information, and it is easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema), the description is reasonably complete: it covers purpose, matching behavior, and enumerates the returned information. The only notable omission is clarifying behavior when a partial title matches multiple datasets, but this does not severely undermine usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the 'title' parameter with examples and the note about distinctive substrings. The tool description adds 'exact or partial title match', but this largely restates the schema's own description. Schema coverage is 100%, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource structure ('Get full details for one dataset on the OCC GIS hub') and clearly enumerates the return contents (distributions, spatial extent, license text). It distinguishes this from sibling tools by focusing on a single dataset found by title, contrasting with list_datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when you need full details of one dataset found by exact or partial title match. It does not explicitly name alternatives or state when not to use it, but the context is unambiguous and implies a targeted lookup rather than listing or querying.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_layer_metadataA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must convey behavioral traits. The verb 'Fetch' implies a read-only operation, and the description lists the returned metadata fields, which is useful. However, it does not disclose error handling, authentication requirements, or any side effects (though likely none). It is adequate but lacks rich behavioral context that might be expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and resource, then lists the metadata fields. It is concise with no fluff, and every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), and the description covers its purpose, input source, and the fields it returns. It goes beyond the minimum by specifying the expected URL type and provenance. Minor gaps include not describing the response format, but the listed fields effectively convey what is returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the single parameter 'layerUrl' with a description and example, giving 100% schema coverage. The description adds a small hint about the URL's provenance (from list_datasets/get_dataset) and that it should be a layer endpoint, but this is marginal. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Fetch') and resource ('ArcGIS FeatureServer/MapServer layer URL'), enumerating the exact metadata fields returned (fields, geometry type, extent, name, description). It also differentiates from siblings by referencing list_datasets/get_dataset and focusing on metadata, not querying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool: for a layer URL such as one returned by list_datasets or get_dataset as 'featureServiceUrl'. This implies it is the follow-up to those tools. It does not explicitly exclude alternatives like query_layer, but the context is clear enough for an agent to choose it for metadata retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_datasetsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the data source URL, search matching behavior (title, description, keywords), and return fields, plus the caveat 'when available' for REST URL and formats, offering honest behavioral insight without promising more than the tool likely delivers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loads the core purpose. It is somewhat dense with comma-separated return field details, but every element serves a functional purpose and no filler exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is present, so the description adequately enumerates return fields and optional filters. It could mention pagination or default limits, but the limit parameter in the schema already covers that, making this sufficiently complete for a list/search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema already has 100% coverage for the two parameters. The description adds meaning beyond the schema by specifying that the query is 'matched against title, description, and keywords', which enriches the schema's generic 'Optional free-text search'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'List/search datasets published on the OCC GIS open data hub', using a specific verb and resource. It clearly distinguishes from siblings like get_dataset (single dataset) and query_layer (feature query) by focusing on dataset listing/searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does and the optional query filter, but it does not explicitly state when to use this tool over siblings or when to avoid it. Usage is implied by the 'List/search' purpose, but no alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_layerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format (JSON), optional geometry inclusion, and the nature of querying (read-only is implied). It could mention error handling or response size limits, but the 'Use get_layer_metadata first' hint and example layer URLs add useful context beyond minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action, and every word contributes. It avoids redundancy while including examples and a usage hint, making it both concise and structured for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters and no output schema, the description covers the essential context: purpose, return type, and a prerequisite action. It does not mention output structure or error handling, but given the interactive schema and sibling context, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds a helpful pointer to get_layer_metadata for field discovery, but does not elaborate on parameter relationships or edge cases. This meets the baseline for high schema coverage without adding substantial extra meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs an attribute and/or spatial query against an ArcGIS FeatureServer/MapServer layer and returns matching features as JSON. It uses a specific verb 'Run' and identifies the resource (layer), examples (OCC well points, tank sites), and distinguishes from siblings by explicitly mentioning get_layer_metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context for when to use this tool (attribute/spatial query) and instructs to use get_layer_metadata first to see available fields, which implies a prerequisite step. It does not explicitly mention when not to use it or compare with list_datasets/get_dataset, but the guidance is clear enough for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.