Server Quality Checklist

Tool Scores

• check_map_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It discloses that the tool checks job completion, downloads when finished, and that redundant generate_map calls create duplicate jobs. However, it does not describe what happens when the job is not finished (e.g., return status) or any side effects of downloading, which prevents a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. The first sentence states the purpose immediately, and the second paragraph delivers crucial usage guidance without fluff. Every sentence adds value, and the text is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters, no annotations, and no output schema, yet the description does not explain parameter meanings or return/result behavior beyond 'download it if so.' It gives strong usage context but leaves significant gaps about what the tool returns and how the optional parameters affect the download, making it incomplete for an agent to use properly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 5 parameters with 0% description coverage, and the description only mentions 'dataKey' conceptually but never explains the parameter names or their roles. The optional parameters (format, color_set, canvas_width, canvas_height) are completely undocumented. The description fails to compensate for the schema's lack of parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check whether a still-in-progress generate_map job has finished, and download it if so.' This identifies a specific verb+resource and differentiates it from siblings like generate_map (which starts jobs) and redownload_map (which presumably re-downloads an existing map).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage constraints: 'Only call this after generate_map reports it is still processing and gives you a dataKey to check.' It also gives timing guidance ('Wait about 20-30 seconds between calls'), warns against tight polling loops, and explicitly says not to call generate_map again for the same location while pending. This is outstanding when-to-use/when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redownload_mapA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description discloses key behavioral traits: map data is NOT regenerated, operation is instant, and it allows changing color theme, format, and canvas size. Could mention error handling for invalid dataKey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a short paragraph and bullet points. Every sentence adds value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and no output schema, the description covers usage context and key behaviors, but omits what the return value is and fails to explain the edge_weight parameter, leaving gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. The description explains data_key implicitly, and color_set/format/canvas dimensions via use cases, but does not mention edge_weight or provide explicit per-parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Re-download a previously generated map' with specific verb and resource, and differentiates from sibling generate_map by noting it uses dataKey and is instant.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists specific use cases (different color, format, canvas size) but does not explicitly state when not to use it (e.g., if the map hasn't been generated beforehand).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compute_routeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of explaining behavior. It discloses the return value (ordered list of coordinates, resolved from/to), notes compatibility with generate_map, and implies a pure computation with no side effects. This goes beyond basic 'computes a route' and covers the key operational aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient: a clear first sentence, return-value summary, usage guidance, and a numbered example flow. Every sentence adds value, and the code snippet concretely demonstrates integration without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of output schema and annotations, the description adequately covers return structure and how to chain with generate_map. It could mention error behavior or mode values, but for a straightforward compute tool the provided information is sufficient for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It illustrates usage with from_location/to_location objects and gives an example with 'label', but it does not explain the 'mode' parameter (walking/driving) or the nested 'location' structure. The example helps but does not fully cover all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Compute a walking or driving route between two locations,' using a specific verb and resource. It distinguishes from siblings by covering route computation rather than map generation or status checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool (pass result to generate_map, use from/to as markers) and gives a typical flow. It does not explicitly mention when not to use it or alternative tools, but the intended workflow is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_mapA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It covers side effects (saves to Downloads), return behavior (dataKey in text result), asynchronous behavior (may return dataKey and ask to check status later), and the consequence of calling again (redundant duplicate job). This is substantial and goes beyond a simple operation description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, using bullet points for the coverage shape and short paragraphs for the async behavior. Every sentence delivers useful information—no fluff or repetition. It is about 100 words, front-loaded with the main purpose, and easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 15 parameters, no annotations, and no output schema, the description provides a strong overview of the tool's operation, including coverage semantics, return value (dataKey), and async flow. It does not explain all parameters, but the essential flow and usage are clear. The description is complete enough for core use cases, though advanced parameters remain undocumented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description compensates for the 0% top-level schema coverage by explaining the coverage shape parameters (default, radius, size_ew + size_ns). However, it does not clarify many other parameters (format, layers, markers, color_set, canvas_width/height). It adds value for the core geometric parameters but leaves significant gaps for the rest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a styled map image for a given location and save it to Downloads.' It uses a specific verb ('Generate') and resource ('styled map image'), and differentiates from sibling tools by describing the generation step, coverage options, and the alternative redownload_map for changing colors/format without regeneration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use and when-not-to-use guidance. It names alternatives: 'Use redownload_map(dataKey=...) to get the same map in a different color or format' and 'call check_map_status(dataKey=...) after a delay instead of returning the image directly — do not call generate_map again for the same location in that case.' This clearly distinguishes this tool from its siblings and prevents redundant duplicate jobs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.