Server Quality Checklist

Tool Scores

• get_city_cost_rangeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds value by specifying the output includes a range, source, date, price basis, and caution, and clarifies it is not a contractor quote. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff. The first sentence delivers the core purpose, the second adds a critical caution. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so the description compensates by describing the output contents (range, source, date, price basis, caution) and the time frame (2026). It covers the key aspects for an agent to understand what to expect, though it could mention units or methodology reference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with all parameters described (metro, state, project). The description does not provide additional parameter semantics beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a 2026 total-project planning range for one city and project type, with source, date, price basis, and caution. It effectively distinguishes from sibling tools like compare_city_costs or list_markets by specifying it returns a single range.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (for a planning range for a specific city and project type) but does not explicitly state when not to use or mention alternatives among sibling tools. The caution 'This is not a contractor quote' provides some guidance on expectations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_marketsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds context about the versioned snapshot but no additional behavioral traits such as rate limits, authorization needs, or what happens if markets change. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy. It front-loads the action and includes essential details (versioned snapshot, city/state level). Every word is necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema. The description adequately explains what the tool returns (list of city and state markets) and its context (2026 snapshot). It could optionally mention the return format, but for a basic listing, the description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema description coverage is 100%. The description does not need to explain parameters. Baseline is 4, and no additional parameter info is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List city and state markets available in Renology's versioned 2026 city-level renovation cost snapshot.' It specifies the verb (list), resource (markets), and scope (city/state, versioned). It also distinguishes from siblings like list_project_types and get_city_cost_range by being a pure listing of available markets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (e.g., before querying costs) but does not explicitly state when to use this tool versus alternatives or provide any exclusions. There is no 'use this when...' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_project_typesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description adds minimal behavioral insight. It provides context ('versioned snapshot') but no unexpected side effects or constraints beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-constructed sentence that conveys all necessary information without unnecessary words. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description fully explains what the tool returns and its scope. No missing information for a simple list retrieval.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100%. The description adds no parameter details because none exist. Baseline 4 is appropriate for no-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists project types within a specific context ('versioned city-level renovation cost snapshot'). This verb-resource pair is distinct from sibling tools like 'list_markets' or cost functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives, but the purpose and sibling names make it implicitly clear that this is for retrieving project type options before using other city cost tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_city_costsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the description adds value by specifying results are planning benchmarks sorted by low end. It provides context on the nature of the output but does not cover error conditions or authorization needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences that front-load the action and add clarifying context. Every sentence provides value with no redundancy or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only two parameters with full schema descriptions, annotations, and no output schema, the description adequately explains the return type (planning benchmarks, sorted by low end). It lacks details on result structure but is sufficient for a read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for both parameters is 100%, so the baseline is 3. The description does not add additional parameter-level details beyond what the schema provides, such as format for metros or project.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares the same 2026 project-type planning range across multiple cities, using specific verbs and resources. It distinguishes from sibling tools like get_city_cost_range (single city) and list_markets (market listing) by focusing on cross-city comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when comparing planning ranges across cities, and explicitly notes results are not quotes or contractor rankings. However, it does not explicitly state when not to use it or name alternatives, but the sibling context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_methodology_and_citationA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the agent knows it's safe. The description adds value by detailing the exact content returned (provenance, dates, URLs, etc.), which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, front-loaded with the purpose and lists all key return items. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explicitly enumerates all return fields. With no parameters and strong annotations, the definition is fully complete for this simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description does not need to add parameter info as the schema is complete with zero properties.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verbs ('Return') and lists concrete items (provenance, dates, limitations, URLs, license, disclosure, citation). Clearly distinguishes from siblings which are list/get operations on markets, project types, and city costs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives, but the tool name and description are self-evident. With zero parameters and a focused purpose, context is clear enough for an agent to select it over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.