Server Quality Checklist

Tool Scores

• get_route_staticA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint, idempotentHint) already indicate safety. Description adds context that data is static and what it includes, but doesn't disclose any hidden behaviors. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence is purposeful. Front-loaded with purpose, then usage guidelines, then parameter notes. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a static data retrieval tool with an output schema (not shown) and sibling tools listed, the description covers all necessary context: what, when, when not, and how to get inputs. Complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the parameter description is clear. Description adds value by explaining how to obtain the route name if needed (via get_stops_around_location), which aids tool selection.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Returns' and lists concrete resources: static route metadata, stop lists, polylines. It distinguishes from siblings by explicitly mentioning when not to use (live positions -> get_route_realtime).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases (user asks for stops, map, scheduled times) and non-use cases (live positions) with alternative named. Also gives prerequisite for obtaining route name if unknown.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stop_realtimeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, and not open-world. Description adds that it produces both a map UI block and a structured arrival list, and requires a numeric stop ID from signage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: a few sentences with bold for emphasis. Every sentence provides value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one well-described parameter, present annotations, and an output schema, the description covers purpose, usage, prerequisites, and behavioral outcome. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds meaning by explaining the stop_id is the municipal stop code shown on signage and accepts integer or digit-only string. Also provides scenario context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it returns live arrivals and vehicle positions, and distinguishes from sibling tools like get_stop_geometry. The verb 'Returns' and resource specification are explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use as default tool for arrivals/departures/vehicles, and when not to use (prefer get_stop_geometry for static polylines). Also advises using get_stops_around_location if only address/coordinates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_route_realtimeA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, idempotentHint=true, openWorldHint=false. Description adds real-time and map rendering context, consistent with annotations. No contradictions, but could mention possible data latency or frequency of updates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus a note, all valuable. Front-loaded with the main action, then usage guidelines and input format. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to explain return values. It covers what the tool does, when to use it, and input requirements. Fully sufficient for an agent to decide to invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'route_name' with 100% schema coverage. Description adds examples ('T30', '32A') and clarifies it accepts short name or numeric external ID, providing more meaning than the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns live positions for all vehicles on a route, optimized for map rendering. It distinguishes from siblings by specifying use cases like 'where is my tram/bus right now?' versus arrival times at a stop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use this tool vs alternatives: prefer get_stop_realtime for arrival times, get_route_static for route shape/stop list without live data. Also specifies input format requirement (route short name or numeric external ID).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stop_geometryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, openWorldHint. The description adds context: 'No live data is fetched', aligns with annotations, and describes output (marker and polylines). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with clear front-loading: purpose first, then usage guidance, then prerequisite/negative. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists, description doesn't need to detail return values. It covers purpose, usage, prerequisites, and contrasts with siblings. Complete for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds value by clarifying parameter requirement ('Requires a numeric stop ID' and guiding on acquiring it via `get_stops_around_location`).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Returns static map context for a stop: its marker and polylines for every route that serves it.' It distinguishes from siblings like `get_stop_realtime` by specifying it does not fetch live data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use (enrich map, visualize routes without live arrivals) and when not to use (live data needed), naming the alternative `get_stop_realtime`. Also mentions prerequisite `get_stops_around_location`.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stops_around_locationA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint and idempotentHint. The description adds that it emits a map UI block for map-capable clients and returns only stop metadata, providing behavioral context beyond annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, and each sentence adds valuable information without redundancy. It is appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description adequately covers purpose, usage, parameter guidance, behavioral traits, and visual output. It is complete for a discovery tool with rich annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with descriptions. The description adds usage guidance for radius_meters (default 1000m, narrow for urban, widen for rural) that complements and adds value beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it discovers transit stops near a point, returning metadata like stop code, name, coordinates, and walking distance. It explicitly distinguishes from sibling tools by saying it's the first step before get_stop_realtime or get_stop_geometry and should not be used for arrivals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use (as first step when user provides location) and when not to use (for arrivals or live data). It names alternative tools for different purposes, providing clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.