Server Quality Checklist

Tool Scores

• get_distanceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It doesn't state what happens for unknown towns, whether the distance is a great-circle/straight-line approximation (beyond the word 'straight-line'), whether output is rounded, or the return format. Positioned as a pure read function, but details are sparse.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence with zero wasted words. It packs the verb, resource, measurement type, and destination into a minimal, scannable format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema and no annotations, the description is thin. It doesn't specify the return unit/format, behavior for invalid input, or how it integrates with sibling tools. While simple, it lacks the behavioral details an agent needs to confidently use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'town' is one town (singular) and that distance is measured to Boston, which adds semantic context. However, it doesn't clarify whether the town name must be a specific canonical form, match sibling tool conventions, or examples of valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('get'), resource ('distance'), and scope ('straight-line distance from one town to Boston'), defining a specific, measurable output. It's distinguishable from siblings (get_housing, get_schools, get_safety) which focus on location features rather than distance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't explain use cases like comparing town proximity or filtering by distance, nor does it exclude any scenarios. The description implies a simple distance lookup but offers no context for when it would be relevant.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_housingB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It confirms this is a read operation (getting median home price), but doesn't disclose data freshness, what time period the median reflects, potential missing-data behavior, or whether the price reflects current listings vs sold values. Without annotations, more behavioral context would be expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, extremely efficient - 'Median home price for one town, in USD.' covers purpose, scope, and unit with zero wasted words. No redundancy with the name or schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool, the description is mostly adequate. There is no output schema, no annotations, and 0% schema coverage, so the description carries the entire burden. It tells the agent the return value conceptually (median home price in USD) but leaves uncertainty about date range, data recency, and edge cases. Given the simplicity of the tool, this is near-minimum-viable but has noticeable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is one parameter (town) with 0% schema description coverage, meaning the schema property has no description beyond its name. The description states the town is a single town (scope), which adds some meaning, but doesn't clarify expected format (e.g., 'Springfield, MA' vs 'springfield'), case sensitivity, or valid range of town names. With only one param, the baseline credit is partially earned but minimal guidance is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Median home price for one town, in USD' clearly states the verb+resource (get median home price) and adds the scope constraint 'one town'. It distinguishes from siblings like get_distance, get_schools, get_safety which cover different domains. However, it doesn't explicitly name the alternative tools, slightly limiting differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage in the context of querying housing cost data for a single town, and the sibling tool names (get_distance, get_schools, get_safety) suggest the broader context of town comparison. However, there are no explicit when-to-use or when-not-to-use instructions, nor any exclusions or alternatives named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_safetyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided at all, so the description carries the full burden of behavioral disclosure. The description states the metric and unit (violent crime rate per 1,000 residents), which is useful, but it does not disclose what happens for unknown towns, whether the data is recent, whether it includes other crime types beyond violent crime, or the reliability/source of figures. For a data-query tool with zero annotation coverage, the behavioral context is thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One crisp sentence, front-loaded with the action and resource, then the unit detail. Zero waste, all content earns its place. This is an example of effective conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no nested objects), the description is fairly complete for the core purpose. However, with no annotations and no output schema, the agent gets no information about response format, error handling for invalid towns, or data vintage. For a simple tool the bar is lower, but the absence of any behavior-disclosure given zero annotation coverage leaves it at 'adequate but with gaps'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the only parameter (town) has no description. Both the description and schema leave 'town' interpretation entirely to the agent. However, the description clarifies that the result is per-1000-residents violent crime rate, which adds meaning about the tool's output framing. Since there is only one obvious parameter whose semantics are inferable from the tool name and description, the gap is minimal — but a note on town name format would help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Get crime figures for one town') and the scope is clear — it provides a specific metric (violent crime rate per 1,000 residents). It distinguishes from siblings by focusing on safety/crime data for a town, though it doesn't explicitly contrast with get_schools, get_housing, etc. The resource and metric are clear but the sibling differentiation is implicit rather than explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (query crime figures for a single town), but doesn't state when to use this vs alternatives, nor does it mention limitations like 'only handles one town at a time' or exclude scenarios. It doesn't say when NOT to use it or point to a sibling for related data. The context signal of a single required 'town' parameter makes usage somewhat self-evident, but explicit guidance on vs-alternatives is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schoolsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, leaving the description to carry the disclosure burden. It does disclose the rating scale (1-10) and source (GreatSchools district rating). However, it doesn't say whether this is a read operation, what happens with invalid town names, or whether it returns per-school or district-level data. Some useful context, but gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single compact sentence with zero wasted words. The parenthetical adds the scale and source detail efficiently. Appropriate length for a simple one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter look-up tool, this is near-adequate. No output schema exists, so the description doesn't explain the return format, but for a rating lookup the '1-10' scale disclosure helps. It could clarify whether output is a single number or a breakdown, but for this complexity level it's reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. The lone parameter 'town' is a string and its meaning is self-evident from the description's 'one town' phrasing. The description confirms the town parameter scope but adds little beyond what the parameter name implies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb+resource+scope: 'School rating for one town' with a specific metric (GreatSchools district rating, 1-10). It's distinguishable from siblings (get_housing, get_distance, get_safety) which cover different domains. The parenthetical adds precision about what kind of rating and its scale.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this returns school ratings scoped to a single town, contrasting with broader alternatives. However, it doesn't explicitly state when to use this vs alternatives or exclude other use cases. The 'one town' scoping is the only guidance, which is modest but helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.