Server Quality Checklist

Tool Scores

• list_servicesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states only that the tool lists services; it does not disclose whether folder scopes results, whether the response is paginated, or what happens when the source lacks a catalog. This is minimal behavioral disclosure for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, and the main scope is front-loaded. However, the brevity partly reflects missing behavioral and parameter detail rather than efficient compression of complete information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With two parameters, no annotations, and a set of sibling tools, the description leaves important gaps: folder semantics, source format, and appropriate usage relative to alternatives. The presence of an output schema covers return-value expectations, but the description is still incomplete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the schema provides no parameter descriptions. The description gives partial semantics for 'source' (a source with an ArcGIS catalog) but says nothing about 'folder,' its default, or its effect on the results. It does not compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('List') and resource ('services published by a source that has a catalog (ArcGIS)'), which distinguishes it from sibling tools like describe_service that focus on a single service or layer. It could be more explicit in contrasting with those siblings, but the core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as describe_service or query_layer. The only contextual hint is 'source that has a catalog (ArcGIS),' which implies a prerequisite but does not explain how to identify such a source or what to do after listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_legendC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals a key behavioral trait: the return format depends on the source type (structured entries for ArcGIS, image for WMS). With no annotations, the description carries the full burden, and it does provide some behavioral context beyond the name and schema. However, it does not disclose other relevant behaviors such as error handling, permissions, or how layer_id affects the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 11-word sentence with no filler, repetition, or unnecessary detail. It is front-loaded with the core concept and immediately specifies the key format difference. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no parameter descriptions, the description is too thin for an agent to reliably invoke the tool. It does not explain the meaning of 'source' or 'layer_id', nor does it clarify when to use this tool relative to siblings like describe_layer or export_map. The presence of an output schema is helpful, but the description alone leaves important gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema description coverage is 0%, and the description adds no meaning for either 'source' or 'layer_id'. The ArcGIS/WMS distinction hints at source behavior, but it does not explain what values are expected or how layer_id influences the result. The tool fails to compensate for the undocumented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states that the tool returns the legend of a source and distinguishes between structured entries for ArcGIS and an image for WMS. This communicates the core resource and output type, and the ArcGIS/WMS distinction helps separate it from sibling tools like describe_layer or query_layer. However, it lacks an explicit verb like 'retrieve' or 'get', relying on the tool name to carry the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as describe_layer or query_layer. The description gives no context about when a legend is needed, no exclusions, and no mention of sibling tools or conditions that would make another tool more appropriate. An agent is left to infer usage entirely from the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• identify_at_pointC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions WGS84 and the return of attributes, implying a read-like operation, but it does not explain tolerance behavior, layer filtering semantics, error conditions, or response structure. This is insufficient for an unpsecified geospatial operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core question, followed by the precise coordinate reference and layer scope. It contains no filler, though its brevity leaves substantial room for richer content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters, no annotations, and no schema descriptions, the description omits important details like tolerance meaning, layers default behavior, and expected output beyond generic 'attributes'. The presence of an output schema helps but does not make up for missing parameter and usage context in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description is the only param guidance. It clarifies that lon/lat are in WGS84 and that it applies to one or more layers of a source, which partially covers source, lon, lat, and layers. However, it entirely ignores tolerance and does not explain how layers are specified or filtered, so the compensation is incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the operation: it returns attributes for a given lon/lat in WGS84 for one or more layers of a source. This distinguishes it from sibling tools like describe_layer or list_services by tying the operation to a spatial point, though it does not explicitly differentiate from query_layer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool instead of query_layer or other siblings. There are no explicit use cases, exclusions, or alternative tool references, leaving the agent to infer when identify_at_point is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_serviceB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It signals a read-only metadata operation and adds a useful nuance: it returns capabilities the source 'actually supports,' not just declared capabilities. It does not discuss auth or errors, but the output schema covers the return shape and this is a benign describe operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It packs the output scope into a concise list, making it easy to scan, though it could have spent a few words on the source parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists, so return values do not need to be fully spelled out. However, the one required parameter is left undocumented and there is no guidance on choosing this tool over closely related siblings, leaving a moderate gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description never explains what 'source' should be — a service ID, URL, or name. The phrase 'this source' weakly ties the parameter to a service, but that is not enough for an agent to reliably construct valid input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly names the resource ('service') and enumerates the metadata contents: layers with ids, extent, CRS, formats, limits, and supported capabilities. This distinguishes it from sibling tools like describe_layer or list_services, though it uses a noun phrase rather than an explicit verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used when service-level metadata is needed, but it does not explicitly state when to prefer describe_service over describe_layer or list_services, nor does it mention any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_layerA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It openly states that WMS sources publish no field schema and that the tool will say so, which is useful behavioral context. It does not discuss errors or permissions, but for an introspection tool with an output schema, the key behavior is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: the first front-loads the core purpose and output, and the second adds the important WMS exception. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with an output schema, the description adequately explains what the tool returns and highlights a source-specific limitation. It is slightly incomplete because it leaves parameter provenance and sibling-tool selection to inference, but those gaps are not severe.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description needed to clarify source and layer_id, but it does not explain their formats, origins, or how they relate to other tools. The mention of 'WMS sources' gives a small hint, but an agent still has little guidance on what values to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb and resource ('Describe a layer') and enumerates the exact return payload: fields with types, coded-value domains, geometry type, and record count. This clearly distinguishes it from siblings like describe_service and query_layer. The WMS caveat further sharpens the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: call this when layer metadata or schema is needed. However, it does not explicitly mention when to prefer this over describe_service, list_services, or get_legend, nor does it state exclusions or selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compose_mapA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full disclosure duty. It explains that bbox is lon/lat, that projection, pixel grid, and legend filtering are handled automatically, and that sources are ordered bottom-first. This meaningfully describes behavior beyond the bare schema and gives the agent operational expectations, though it stops short of describing output format or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and no filler. The first sentence states the operation; the second packs the essential input semantics. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter tool with no annotations and no output schema, the description covers the non-obvious aspects: named compositions, source ordering, legend modes, and automatic projection/pixel-grid handling. It does not spell out whether composition and sources may be combined, nor the exact returned format, but these are minor gaps given the strong operational guidance provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains bbox coordinates, the composition-or-sources choice, bottom-first ordering, and legend_mode values with a use recommendation for 'separate'. Width, height, and title are not deeply explained, but the statement about pixel-grid handling and the param names provide enough orientation for a capable agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific operation and resource: 'Compose one map from several sources, aligned, with a combined legend.' This clearly distinguishes compose_map from sibling tools like query_layer, get_legend, and export_map by framing it as multi-source assembly. It also names the two input modes (composition or sources), removing ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: use a declared `composition` or list `sources` bottom-first, and choose `legend_mode` with a specific recommendation for geological keys. It does not explicitly name alternatives or state when not to use the tool, but the intended use case is clear enough for an agent to route correctly among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_mapA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it does so reasonably: it discloses the output form (image), the coordinate system (WGS84), and the bbox ordering. It does not mention potential output format details, layer handling, or error behavior, but there is no contradiction or hidden destructive behavior implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two short sentences, no filler, with the primary behavior and output type stated first and the routing alternative stated second. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description covers the essential invocation context: what to render, the bbox format, and how it differs from compose_map. The main gap is lack of explanation for the optional layers, width, and height parameters, but schema defaults and names mitigate that gap enough for an agent to make a competent call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema description coverage is 0%, the description needs to compensate. It usefully explains source and bbox semantics, including coordinate order and units. However, optional parameters like width, height, and layers are not described, so the agent must infer their meaning from their names and schema defaults. This is adequate for required parameters but incomplete for optional ones.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Purpose clarity is excellent. It uses a specific verb ('Render'), names the resource ('one source'), defines the spatial extent ('bbox [minlon,minlat,maxlon,maxlat]'), and states the return type ('image'). It also distinguishes itself from the sibling compose_map by explicitly noting the single-source scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear when-to-use guidance: use this tool for a single source in a bbox, and explicitly directs the agent to compose_map for multiple sources. The bbox coordinate order is also given, leaving little ambiguity about how to construct the required parameter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_layerA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does well: it discloses the output CRS (WGS84), the pagination pattern via exceededTransferLimit and offset, and the source capability prerequisite. It does not deeply discuss errors or side effects, but the query nature makes those less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: first sentence states purpose and output, second sentence lists capabilities, third gives pagination guidance, fourth gives source constraint. Every sentence earns its place without repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 9-parameter complexity, the description covers the essential behaviors: filtering, output format, coordinate system, pagination, and capability prerequisite. The output schema exists and can carry return-value details. Minor gaps remain around limit semantics and return_geometry behavior, but overall context is solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It meaningfully explains several parameters: where (SQL), out_fields ('field selection'), order_by ('ordering'), bbox (with coordinate format), and offset (pagination). It does not detail limit or return_geometry, but covers the most important semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific operation ('Query layer attributes'), the output format ('GeoJSON in WGS84'), and core controls (where, field selection, ordering, bbox). It clearly differentiates this from sibling metadata/rendering tools such as describe_layer and export_map by focusing on data-retrieval semantics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: querying layer data with SQL-like filtering, bbox, and pagination. It does not explicitly enumerate alternatives or say 'use X instead', but the intended use case is strongly implied by the capability constraint 'Only sources that declare the query capability'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.