Server Quality Checklist

Tool Scores

• mapbox_directionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Get navigation route' implies a read-only operation, it doesn't disclose important behavioral traits like authentication requirements, rate limits, response format, error conditions, or whether this is a real-time vs. cached service. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence that gets straight to the point with zero wasted words. It's front-loaded with the core functionality and uses clear, direct language without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what the tool returns (route geometry? distance? duration? turn-by-turn instructions?), doesn't mention error conditions or limitations, and provides no context about the Mapbox service it interacts with. The agent would have significant gaps in understanding how to properly use this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters well-documented in the schema itself. The description doesn't add any meaningful parameter semantics beyond what's already in the schema - it mentions 'between two points' which implies the coordinates array should contain exactly two coordinate objects, but this constraint isn't explicitly stated in either the description or schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('navigation route between two points'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'mapbox_directions_by_places', which likely serves a similar purpose with different input parameters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of sibling tools like 'mapbox_directions_by_places' or 'mapbox_matrix', nor any context about when coordinate-based routing is preferred over place-based routing or matrix calculations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_directions_by_placesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions geocoding implicitly ('using their names'), but doesn't clarify authentication needs, rate limits, error handling, or what the route output includes (e.g., distance, duration, geometry). For a tool that performs geocoding and routing, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality. Every word earns its place, with no redundant or vague phrasing, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool with geocoding and routing complexity. It doesn't explain what the route output includes (e.g., JSON structure, key fields like distance or polyline), error cases (e.g., invalid place names), or behavioral constraints, leaving gaps for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters. The description adds minimal value beyond the schema—it implies 'places' are names (not coordinates) and mentions geocoding, but doesn't explain parameter interactions or usage nuances. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get navigation route') and resource ('between multiple places'), specifying it uses place names rather than coordinates. However, it doesn't explicitly differentiate from sibling tools like 'mapbox_directions' (which likely uses coordinates) or 'mapbox_matrix_by_places' (which might calculate multiple routes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention sibling tools like 'mapbox_directions' (for coordinate-based routing) or 'mapbox_matrix_by_places' (which might handle distance matrices between places), leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_geocodingC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions 'search' and 'convert' but doesn't disclose rate limits, authentication needs, error handling, or what the output looks like (e.g., coordinates format). This is inadequate for a tool with 5 parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for the tool's complexity, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no annotations, no output schema), the description is incomplete. It lacks behavioral details, usage context, and output information, leaving significant gaps for an agent to operate effectively without trial and error.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no additional meaning beyond implying geocoding functionality, which aligns with the schema but doesn't enhance parameter understanding. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('search for places' and 'convert addresses into coordinates') and identifies the resource (geographic data). It distinguishes from sibling tools like mapbox_directions by focusing on geocoding rather than routing, though it doesn't explicitly name the differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like mapbox_directions or mapbox_matrix. It doesn't mention use cases, prerequisites, or exclusions, leaving the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_matrixC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic function without addressing critical aspects. It doesn't mention whether this is a read-only operation, potential rate limits, authentication requirements, error handling, or what the output looks like (e.g., matrix format). For a complex tool with 9 parameters, this lack of behavioral context is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently communicates the core functionality without any wasted words. It's appropriately sized and front-loaded, making it easy for an agent to quickly grasp the tool's purpose while leaving detailed parameter information to the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 9 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain the output format (e.g., what the matrix looks like), error conditions, performance characteristics, or how it differs from sibling tools. The agent would need to rely heavily on trial and error or external knowledge to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information beyond what's already in the schema, which has 100% coverage with detailed descriptions for all parameters. While the description implies the tool uses coordinates and calculates matrices, it doesn't explain parameter interactions, defaults, or practical usage examples. Given the high schema coverage, a baseline score of 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('calculate') and resource ('travel time and distance matrices between coordinates'), making it immediately understandable. However, it doesn't distinguish this tool from its sibling 'mapbox_matrix_by_places', which likely serves a similar purpose with different input methods, leaving room for improvement in sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'mapbox_matrix_by_places' or other siblings such as 'mapbox_directions'. There's no mention of prerequisites, constraints, or typical use cases, leaving the agent with insufficient context to make informed decisions about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapbox_matrix_by_placesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions what the tool does but lacks critical behavioral details: it doesn't specify rate limits, authentication requirements, error handling, or what happens when place names are ambiguous. For a tool with 8 parameters and no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core functionality without unnecessary words. It's front-loaded with the main purpose and uses clear terminology. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no output schema, no annotations), the description is incomplete. It covers the basic purpose but lacks context on behavioral traits, usage guidelines, and output format. While the schema handles parameter documentation well, the description doesn't compensate for the absence of annotations or output schema, leaving gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema—it implies the 'places' parameter uses names rather than coordinates, but doesn't explain parameter interactions or provide examples. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Calculate travel time and distance matrices between places using their names'. It specifies the verb ('calculate'), resource ('travel time and distance matrices'), and method ('using their names'). However, it doesn't explicitly differentiate from sibling tools like 'mapbox_matrix' (which likely uses coordinates rather than place names), leaving room for improvement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'mapbox_matrix' (which might use coordinates) or 'mapbox_directions_by_places' (which might provide route details rather than matrices), nor does it specify prerequisites or exclusions. The agent must infer usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.