Server Quality Checklist

Tool Scores

• mapbox_directionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get navigation route' implies a read-only operation, it doesn't address authentication requirements, rate limits, error conditions, response format, or whether this is a real-time vs. cached service. The description provides minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just 5 words, front-loading the core purpose without any wasted language. Every word earns its place in communicating the essential function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a navigation API tool with no annotations and no output schema, the description is insufficient. It doesn't explain what a 'navigation route' includes (distance, duration, steps, geometry), error handling, authentication needs, or how results differ from sibling tools. The minimal description leaves too many contextual gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents both parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain coordinate ordering (origin to destination), minimum/maximum points, or practical usage of the profile parameter. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('navigation route between two points'), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'mapbox_directions_by_places', which likely serves a similar purpose with different input parameters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'mapbox_directions_by_places', 'mapbox_geocoding', and 'mapbox_matrix' available, there's no indication of when this coordinate-based directions tool is preferred over place-based or matrix alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_directions_by_placesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It mentions geocoding ('using their names') which implies place name resolution, but doesn't disclose rate limits, authentication needs, error handling, or what the route output includes (e.g., distance, duration, geometry). For a navigation tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, with every word contributing to understanding the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a navigation routing tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the route output contains (e.g., waypoints, instructions, polyline), performance characteristics, or common failure modes. The geocoding aspect is hinted at but not elaborated, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description adds no additional parameter semantics beyond implying that 'places' are resolved via geocoding. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get navigation route') and resource ('between multiple places using their names'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'mapbox_directions' or 'mapbox_matrix_by_places', which likely have overlapping functionality with different input approaches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'mapbox_directions' (which might use coordinates instead of place names) or 'mapbox_matrix_by_places' (which might handle multiple routes). There's no mention of prerequisites, limitations, or typical use cases beyond the basic functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_geocodingC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the core functionality but omits critical details such as rate limits, authentication requirements, error handling, response format, or whether this is a read-only operation. For a geocoding tool with external API dependencies, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence) and front-loaded with the core purpose. Every word earns its place, with no redundant or unnecessary information, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (geocoding with 5 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like API constraints, response structure, or error conditions, which are essential for proper tool invocation in a real-world context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, providing clear documentation for all 5 parameters. The description adds no additional parameter semantics beyond what's in the schema, so it meets the baseline score of 3. It doesn't explain how parameters interact or provide usage examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Search for places' and 'convert addresses into coordinates') and identifies the resource (geographic data). It distinguishes from sibling tools by focusing on geocoding rather than directions or matrix calculations, though it doesn't explicitly name those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools (mapbox_directions, mapbox_directions_by_places, mapbox_matrix, mapbox_matrix_by_places). It doesn't mention use cases, prerequisites, or exclusions, leaving the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_matrixC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions what the tool does but fails to describe critical behaviors such as rate limits, authentication requirements, error handling, or output format. For a tool with 9 parameters and no output schema, this leaves significant gaps in understanding its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (9 parameters, no annotations, no output schema), the description is insufficient. It doesn't explain the return values, error conditions, or practical usage scenarios. For a matrix calculation tool with multiple configuration options, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any meaningful semantic context beyond what the schema provides, such as explaining relationships between parameters or typical use cases. The baseline score of 3 reflects adequate but minimal value addition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('calculate') and resource ('travel time and distance matrices between coordinates'), providing a specific purpose. However, it doesn't differentiate from sibling tools like 'mapbox_matrix_by_places', which likely serves a similar function with different input types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'mapbox_matrix_by_places' or other sibling tools. The description lacks context about use cases, prerequisites, or exclusions, offering only a basic functional statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_matrix_by_placesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions what the tool does but doesn't describe important behavioral traits: whether it requires authentication, rate limits, what happens when place names are ambiguous, error handling, or the format of returned matrices. For a complex tool with 8 parameters, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that clearly states the tool's core functionality. It's appropriately sized and front-loaded with the essential information. There's no wasted verbiage or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex matrix calculation tool with 8 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the output looks like (matrix format), doesn't mention authentication requirements, doesn't discuss rate limits or costs, and doesn't provide context about when this tool is appropriate versus alternatives. The description should do more given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'using their names' which hints at the 'places' parameter, but doesn't provide additional context about parameter interactions or usage patterns. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Calculate travel time and distance matrices between places using their names.' It specifies the verb ('calculate'), resource ('travel time and distance matrices'), and method ('using their names'). However, it doesn't explicitly differentiate from sibling tools like 'mapbox_matrix' (which likely uses coordinates rather than place names).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'mapbox_matrix' (which might use coordinates instead of place names) or 'mapbox_directions_by_places' (which might provide route details rather than matrices). There's no context about appropriate use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.