Server Quality Checklist

Tool Scores

• check_cold_climate_performanceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the output in detail but does not explicitly state whether the tool is read-only or has side effects. Given the analysis nature, it implies safety, but lacks explicit behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with bullet points and includes necessary detail. It is not overly verbose, but could be slightly more concise. The front-loading with the core purpose is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and explains return values, the description adequately covers the tool's function, inputs, and analysis steps. It is comprehensive for a complex verification tool, leaving no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all 5 parameters, so the baseline is 3. The description does not add significant meaning beyond the schema; it lists what is analyzed but does not elaborate on parameter specifics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Verify') and resource ('heat pump performance at design temperature') and clearly lists what it analyzes and determines. It distinguishes itself from sibling tools like 'calculate_heat_pump_sizing' by focusing on cold-climate performance and backup heat requirements.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Critical for cold-climate installations to ensure adequate heating on coldest days,' which provides clear context for when to use. However, it does not explicitly state when not to use or list alternatives, missing an opportunity for full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_energy_costsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It details output structure but omits behavioral traits like data sources, limits, or assumptions beyond mentioning 'calculation_notes'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with bullet points, front-loaded purpose, and every sentence adds value without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, 4 required, and a full output schema described, the description is thorough. Minor omission: no mention of rate defaulting or fallback behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, and description adds context by grouping parameters into analysis steps, enhancing meaning beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool estimates annual electricity costs and payback period for heat pump vs current system, and lists detailed analysis areas. It distinguishes from sibling tools like sizing calculators.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly conveys usage for cost comparison, but lacks explicit guidance on when not to use or alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_multi_zone_sizingA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description fully explains inputs and outputs including zone structure and return values, but does not mention computational constraints or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear breakdown of zone specification and return results, though somewhat lengthy; no wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists (not shown) but description provides full return structure. For a complex multi-zone tool, covers essential inputs and outputs adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters with descriptions, but zones parameter schema is minimal (additionalProperties: true). Description adds extensive detail on zone sub-fields, compensating for schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it calculates heating/cooling loads for multi-zone homes with detailed zone-by-zone analysis, distinguishing from siblings like calculate_heat_pump_sizing by specifying multi-zone focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides detailed zone specification instructions but lacks explicit guidance on when to use this tool versus siblings or prerequisites (e.g., require construction details).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_electricity_rateA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that it fetches from an external API (EIA), requires an API key, and returns a dictionary with specific fields. It does not mention error handling or rate limits, but for a simple read operation, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two main sentences followed by bulleted return fields. It is front-loaded with the core purpose and contains no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter, no annotations, and an output schema present, the description is complete. It explains the data source, prerequisite, and return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the one parameter 'zip_code' with a clear description. The description adds no extra detail beyond 'ZIP code location', so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets the current electricity rate for a specific ZIP code, using a verb (get) and resource (electricity rate). It distinguishes from sibling tools which are all about heat pump sizing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the requirement of EIA_API_KEY environment variable as a prerequisite, but does not explicitly state when to use this tool versus alternatives or when not to use it. Usage is implied by the context of electricity rates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_heat_pump_modelsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It transparently describes the return structure and filter behavior (e.g., case-insensitive partial match for brand, typical HSPF2 range). No side effects are expected, and the description adequately covers the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for overview, return values, and filter details. It is concise but could be slightly tightened (e.g., some redundancy between filter list and explanation). Still, it is efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description sufficiently explains the return structure (total_models, brands, models). It covers the essential aspects for a listing tool. No major gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful context beyond the schema (e.g., 'Case-insensitive partial match' for brand, 'Capacity range for sizing' for BTU, 'typical range: 8.0-12.0' for min_hspf2), enhancing usability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists heat pump models with optional filters, using a specific verb ('List') and resource ('heat pump models'). It distinguishes from siblings like calculate_heat_pump_sizing by focusing on retrieval rather than calculation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool and details each filter with examples. While it doesn't explicitly state when not to use it, the sibling context provides differentiation. The filter descriptions are clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_heat_pump_sizingA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It fully explains the calculation factors (climate zone via ZIP, square footage, build year, optional humidity) and lists all return fields, making the tool's behavior transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with bullet points and a clear return list. Each sentence adds value, and the main purpose is front-loaded. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description adequately covers inputs and outputs for a sizing tool. It includes optional humidity considerations and detailed return fields. However, it could mention that the calculation is approximate (quick sizing) or any limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, but the tool description adds meaningful context beyond basic schema descriptions (e.g., explains how ZIP code determines climate zone and design temperature). This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it calculates required BTU capacity for a single-zone heat pump installation. It lists specific inputs (ZIP code, square footage, build year, humidity) and outputs, distinguishing it from sibling 'calculate_multi_zone_sizing'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description clearly indicates when to use the tool (for single-zone sizing) but does not explicitly state when not to use or mention alternatives. The sibling list provides context, but the description lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.