Server Quality Checklist

Tool Scores

• get_safety_gradeC
  2.1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits itself. It does not state whether the operation is read-only (though the name implies a get), whether any authentication or special permissions are needed, or any rate limits or error conditions. The only behavioral hint is that lower grades are safer, which is more about output interpretation than tool behavior. This is a significant gap given the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff, so it is concise. However, it is under-specified for a tool with three parameters and an output schema—it omits essential guidance about usage and parameters. The 'lower grades are safer' note is useful but does not make up for the lack of detail elsewhere. It is not inappropriately long, but it is too short to be effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a moderately complex schema (3 params, enum, default) and an output schema, yet the description only conveys the general concept of a safety grade. It does not explain how to interpret the region or category, nor when to use the tool. While the output schema may cover return values, the description still needs to provide enough context to call the tool correctly, which it does not. The absence of any usage context makes it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description adds no information about the parameters. The schema includes 'region', 'category' (with an enum), and 'publication_year', but the description does not explain what 'region' signifies, what each category means, or how publication_year affects the grade. The description fails completely to compensate for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states an explicit verb ('Return') and a clear resource ('safety grade'), and 'official relative' adds valuable context. It is distinct from the sibling tools, which focus on nearby amenities or demographic data. However, it does not specify the form of the grade (e.g., letter, numeric scale) or what 'relative' compares against, leaving some ambiguity about the exact output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the alternatives. The description names no conditions, prerequisites, or exclusions, and gives no indication of which scenarios call for a safety grade rather than, say, nearby hospitals or car-free neighborhoods. The agent is left to infer applicability entirely from the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_car_free_neighborhoodsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior itself. It mentions ranking and filtering by hospital/bus constraints but does not explain what 'pass' means, how ranking is ordered (e.g., by score, access time), what happens when no areas pass, or whether it performs any aggregation. For a tool with no annotations, this leaves significant behavioral uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence with no verbosity or redundancy, and it fronts the main action ('Rank'). However, it is so terse that it sacrifices informativeness; it is concise but under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no schema descriptions, no annotations, and only a one-line description, the tool is insufficiently specified for an agent to call it correctly. The description fails to explain the role of the parameters, the meaning of the result, or how the constraints interact, despite having an output schema and nontrivial parameter defaults. The description carries too little context for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must illuminate the parameters. It vaguely references 'hospital and bus constraints' but does not map them to specific parameters like bus_max_walk_minutes or minimum_daily_bus_trips. The meaning of region, service_day, result_limit, and candidate_limit is entirely unaddressed. The description adds essentially nothing beyond what the schema names already show.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Rank') and a specific resource ('market-centered areas') with explicit constraints (hospital and bus). It distinguishes from siblings like search_nearby_hospitals or search_nearby_markets, which retrieve raw data, whereas this ranks a filtered set. Some ambiguity remains about what 'market-centered' precisely means, but the core purpose is identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus the sibling search tools. It does not indicate, for example, that this tool should be used to get a ranked shortlist after the user has identified areas of interest, nor does it mention any prerequisites or exclusions. The description offers no context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_age_population_ratioC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden of behavioral disclosure. It adds one useful trait beyond the schema: age bounds are inclusive. But it does not disclose what happens when age_from exceeds age_to, whether as_of:null means 'latest available data,' what data vintage is used, or the behavioral meaning of the defaults (70–79, implying the default demographic is older residents).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no filler; the verb, scope, and inclusivity qualifier are all up front. Every word earns its place. This is appropriately sized for a description, even if it is too brief to carry everything the tool needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values need not be described, but the input side is under-covered: 4 parameters with 0% schema descriptions, no annotations, and no mention of region semantics, as_of meaning, or the significant 70–79 default age range. An agent would likely have to guess at valid inputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it barely does. 'Inclusive age range' modestly clarifies that age_from/age_to endpoints both count, but there is zero guidance on how to identify a valid region or what as_of (YYYYMM) represents — two of the four parameters are effectively undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Return') and a specific resource ('resident-population share for an inclusive age range'). The phrase is mostly clear, though the denominator of the 'share' (share of what?) is left to inference. It implicitly distinguishes itself from the sibling search/recommendation tools, but never names an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use, when-not-to-use, prerequisites, or alternative routing is provided. The only reason an agent would pick this over the siblings is the tool's name and the generic purpose statement; no explicit guidance reduces the risk of mis-selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nearby_bus_stopsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It conveys that the search is read-only and that an 'estimated' service threshold is involved, but it does not explain how 'nearby' is measured, whether results are limited, which service day is used, or how the daily-service threshold is applied. This leaves important behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler or redundant restatement of the tool name. The core action is front-loaded ('Find nearby stops') and the distinguishing filter is appended concisely. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only search tool with an output schema and self-explanatory parameter names, the description is minimally adequate. However, it fails to clarify the meaning of the filtering threshold, how walking distance factors into 'nearby', or the relevance of service_day. An agent could invoke it correctly using the schema, but would still be guessing about intended usage semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters, but it does not. 'Daily-service threshold' loosely maps to minimum_daily_trips, and 'nearby' maps to max_walk_minutes, but the required latitude/longitude and the optional service_day are not mentioned at all. The description adds minimal semantic value over the parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb and resource: 'Find nearby stops' and adds a specific filtering criterion ('meeting an estimated daily-service threshold'). It is distinguishable from sibling tools like search_nearby_hospitals and search_nearby_markets because of the 'stops' subject, though 'stops' is slightly ambiguous without the tool name and no sibling is named explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for finding transit stops near a location, but it does not state when to use it versus alternatives, nor does it provide any exclusion criteria. Since the sibling tools cover clearly different resource types (hospitals, markets, neighborhoods, demographics), the lack of explicit routing is less harmful, but guidance is still only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nearby_hospitalsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It does disclose a useful limitation: results are based on an 'estimated straight-line walking time' rather than actual routing. However, it does not describe result limits, ordering, or other operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no wasted words. Every part earns its place, and the core action plus key constraint is communicated immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and the required latitude/longitude expressed in the input schema, the core invocation is reasonably reachable: provide a location and optional walking time. However, optional filter semantics and broader tool-selection context are left incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate by clarifying parameters. It only loosely relates to max_walk_minutes via 'walking time' and does not explain latitude/longitude as the search origin, department_code, or include_departments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Find') on a specific resource ('hospitals') and adds a clear constraint: estimated straight-line walking time. This is sufficient to distinguish it from sibling tools like search_nearby_bus_stops and search_nearby_markets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to choose this tool over alternatives, nor does it mention any exclusions or preconditions. The agent must infer its usage purely from the resource type and sibling tool naming.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nearby_marketsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It does disclose that markets are 'registered' and that walking time is 'estimated,' which is useful. However, it does not state whether this is a read-only operation, whether results are sorted or limited, or whether there are any rate limits or data-source caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler. It front-loads the core action and subject, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a geospatial search tool with four parameters and zero parameter-level schema descriptions, this is too sparse. The output schema likely covers returns, but the description does not clarify region format, coordinate interpretation, or walk-time semantics. An agent would still have to guess at important invocation details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate by explaining parameters. It only hints at max_walk_minutes through 'walking time,' and says nothing about what region should contain, how latitude/longitude define the search center, or how max_walk_minutes is applied. Required parameters and their semantics are left mostly unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Find registered traditional markets within an estimated walking time.' It clearly distinguishes this tool from sibling tools like search_nearby_hospitals and search_nearby_bus_stops by naming the exact subject matter and the walking-time constraint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when the user wants nearby traditional markets reachable by walking. However, it gives no explicit guidance about when not to use it or how it compares to the sibling search_nearby_* tools. The usage context is clear from the wording, but exclusions and alternatives are left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.