Server Quality Checklist

Tool Scores

• arcgis_queryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states the tool returns structured rows without geometry and can filter/limit, but does not disclose side effects, permission requirements, error handling, or read-only nature. Adds some context beyond schema but lacks comprehensive behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with clear front-loading of purpose, followed by key features (filtering, limiting). No redundant words; every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, description partially explains return structure (structured rows, no geometry) but lacks details on row format, field selection, error handling, or pagination. Adequate for a simple query tool but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with adequate descriptions. The tool description adds minor value by mentioning geometry exclusion, but mostly rephrases schema info. Baseline 3 applies as schema already provides sufficient parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (query), resource (feature layer attributes), and outcome (structured rows without geometry). It distinguishes from sibling tools like arcgis_run_gp and arcgis_export_layout by focusing on attribute queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description mentions filtering with 'where' and limiting rows, implying when to use options. However, it does not explicitly state when to use this tool versus siblings, though sibling tools are distinct enough to avoid confusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• arcgis_export_layoutA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors: the export runs in the user's open Pro instance, is visible to the user, and returns output path and file size. No annotations exist, so the description carries the burden and does so well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and contains no superfluous information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the execution context (live Pro instance, user visibility) and return values, which compensates for the lack of output schema. It is complete for a simple export tool, though prerequisites could be mentioned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter described. The description does not add new parameter-level details beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool exports a layout from a live ArcGIS Pro project to PDF, distinguishing it from sibling tools that perform other tasks like pinging, querying, or running geoprocessing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for exporting layouts when a Pro instance is open, but does not explicitly state when to use it versus alternatives or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• arcgis_pingA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It implies a safe read operation but does not clarify edge cases (e.g., no project open, network requirements) or side effects. Minimal transparency beyond stated purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, well-structured sentence in Chinese that front-loads the purpose and expected outputs. No waste, every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter tool with no output schema, the description covers purpose, return types, and usage context. Lacks details on error conditions or output format, but sufficient for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (no parameters), baseline is 3. Description adds no parameter info but correctly reflects the lack of parameters. No extra semantics needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'read' and resource 'ArcGIS Pro project status', listing specific return fields (project name, maps, layouts, active map/layout). It distinguishes from sibling tools which are action-oriented (export, query, run, symbology, zoom).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to use this tool before other commands to understand the open project. It could be improved by noting when not to use it or mentioning it is safe/read-only, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• arcgis_run_gpA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that output layers are automatically added to the map, warns that layer names are unreliable and recommends full paths, and explains the allow_delete safety mechanism. It does not cover error handling or performance, but is still quite transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that efficiently conveys the main purpose first, then details. Every sentence adds value, though a more structured format (e.g., bullet points) could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately covers what the tool does and how to use it. It mentions output visibility and parameter format. It does not describe return values, but the tool's side effect (adding layers to map) is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds value beyond the schema by providing examples (e.g., 'analysis.Buffer'), explaining that params are positional strings, and giving formatting hints like '500 Meters' for distances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs any ArcGIS geoprocessing tool (entire ArcToolbox) and that output layers are added to the current map. It distinguishes itself from siblings like arcgis_query or arcgis_symbology by focusing on arbitrary GP tool execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides examples of tool naming and parameters, and includes a safety opt-in for destructive tools. However, it does not explicitly state when not to use this tool or mention alternatives, which would strengthen guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• arcgis_zoom_toA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that the tool causes visual map movement and can optionally select features first. Without annotations, description carries full burden and adequately covers the visible effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences covering main action, optional behavior, and prerequisite. No wasted words, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool (2 params, no output schema), description adequately explains both usage modes and a key prerequisite. Could mention error cases but not necessary for a zoom tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (baseline 3), but description adds extra detail: explains the where behavior as 'first selects matching features then zooms to the selection', which adds semantic value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb ('zoom to') and resource ('active map view to a feature layer'). It distinguishes from sibling tools which have different purposes (export, query, run geoprocessing, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit prerequisite ('requires Pro to be in map view, not layout') and explains the optional where condition behavior. Does not explicitly contrast with siblings but contextually differentiation is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• arcgis_symbologyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that changes appear immediately in the Pro window, and explains parameter behavior (e.g., 'classes' only for graduated). However, it doesn't mention error handling or permanence of changes, but the main behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences: first states purpose, second explains renderer types, third gives an example. No fluff, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters and no output schema, the description covers the tool's purpose and parameter usage well, including an example. It lacks details on potential errors or prerequisites (e.g., field must exist), but overall it is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant value beyond schema descriptions: it explains the purpose of 'renderer' with examples, clarifies that 'classes' and 'method' apply only to 'graduated', and gives a concrete example. This enhances understanding beyond the parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('套用渲染器' apply renderer) and the resource ('要素图层' feature layer), with immediate visual feedback. It distinguishes between two renderer types ('graduated' and 'unique'), making the purpose precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each renderer: 'graduated' for numeric fields and 'unique' for categorical fields. While it doesn't explicitly exclude siblings, the sibling tools (e.g., export, query) are clearly different, so usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.