Server Quality Checklist

Tool Scores

• get_area_statsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (price statistics with breakdowns) and mentions default values and optional parameters, which adds useful context. However, it doesn't disclose important behavioral traits like rate limits, authentication requirements, error conditions, or whether this is a read-only operation (though 'Get' implies it likely is).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by return details and scope. The parameter explanations are organized in a clear Args section. Every sentence adds value, though the structure could be slightly more polished (e.g., combining the scope mention with the purpose).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which handles return values), 3 parameters with good semantic coverage in the description, and no annotations, the description is reasonably complete. It explains what the tool does, what it returns, and parameter meanings. The main gap is lack of behavioral context (rate limits, errors, etc.) and usage guidance relative to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant semantic value beyond the input schema, which has 0% description coverage. It explains what each parameter means: 'location' accepts postcode, ZIP code, or area name; 'market' is optional and auto-detected; 'months' is look-back period with default of 12. This compensates well for the schema's lack of descriptions, though it doesn't specify format constraints or valid ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get price statistics for a property market area' with specific details about what it returns (median, average, min, max prices with breakdown by property type) and scope ('Covers all 16 markets'). It distinguishes itself from siblings by focusing on aggregated statistics rather than listing markets or searching individual properties. However, it doesn't explicitly contrast with sibling tools in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools (list_markets, search_property_comps). It mentions the scope ('Covers all 16 markets') but doesn't explain when this statistical analysis is preferred over listing markets or searching property comps. There are no explicit when/when-not statements or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_property_compsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns recent sales with specific fields (price, date, etc.) and covers 16 markets, which adds useful context. However, it lacks details on permissions, rate limits, error handling, or data freshness, leaving behavioral gaps for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by market coverage, return data, and parameter details. It's appropriately sized, but the parameter section is lengthy; however, each sentence earns its place by adding critical information given the low schema coverage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, 0% schema coverage, no annotations) and the presence of an output schema, the description is largely complete. It covers purpose, markets, returns, and parameter semantics. A slight gap exists in behavioral details like error cases or performance limits, but the output schema likely handles return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds significant meaning beyond the schema by explaining each parameter's purpose, providing examples (e.g., location formats), listing market codes, specifying defaults, and detailing property type codes per market, effectively documenting all 6 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for comparable property sales near a location, specifying the verb 'search' and resource 'comparable property sales'. It distinguishes from sibling tools like 'get_area_stats' and 'list_markets' by focusing on sales data rather than statistics or market listings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by listing the 16 covered markets and indicating it's for finding recent sales data. However, it doesn't explicitly state when to use this tool versus the sibling tools 'get_area_stats' or 'list_markets', missing explicit alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_marketsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates this is a read-only operation (listing markets) and specifies the output includes transaction counts and location examples, adding useful context. However, it lacks details on potential limitations, such as data freshness or any access restrictions, which would be beneficial for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action ('List all 11 available property markets') and includes essential details without waste. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, but with an output schema), the description is complete enough. It clearly states the purpose and output details (transaction counts and location examples), and since an output schema exists, it does not need to explain return values further. This covers the necessary context for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the schema fully documents the absence of parameters. The description does not add parameter-specific information, but since there are no parameters, the baseline is 4. It effectively communicates that no inputs are needed, aligning with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all 11 available property markets') and includes what information is provided ('with transaction counts and location examples'), distinguishing it from siblings like 'get_area_stats' and 'search_property_comps' which focus on statistics and property comparisons respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it lists 'all 11 available property markets,' suggesting it's for obtaining a comprehensive overview rather than filtered results. However, it does not explicitly state when to use this tool versus alternatives like 'get_area_stats' or 'search_property_comps,' missing explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.