Server Quality Checklist

Tool Scores

• generate_listingA
  3.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description provides exhaustive behavioral details: side effects (read-only, no side effects), authentication requirements, rate limits, error handling, idempotency, and data privacy. This goes well beyond minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections, but contains redundancy: the 'Behavior' and 'Behavioral Transparency' sections overlap significantly, repeating information about side effects, rate limits, and authentication. Could be more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers most aspects: parameter descriptions, behavior, auth, rate limits, error handling. Missing output format details (e.g., what the returned listing looks like) and no output schema, but otherwise thorough for a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The Args section describes most parameters with meaningful details (e.g., enum values for property_type and style). However, the 'api_key' parameter is missing from the description despite being in the schema. Schema coverage is 0%, so the description compensates well but has a gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool generates a property listing. However, the 'When to use' section describes 'structured analysis or classification of inputs', which misaligns with the actual purpose of generating a listing, causing confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Has explicit 'When to use' and 'When NOT to use' sections, but the 'When to use' is generic and doesn't differentiate from sibling tools like analyze_neighborhood or calculate_mortgage. It also fails to mention alternative tools for specific scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_mortgageA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully covers behavioral traits: it states the tool is read-only, stateless, idempotent, has rate limits (free/pro tiers), requires no authentication for basic usage, and outlines error handling and data privacy. This is thorough and leaves no ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Behavior, Transparency). It is somewhat lengthy but every section adds value. The key information is front-loaded: the first sentence states the purpose. Minor redundancy in the 'Behavioral Transparency' subsection could be trimmed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description does not specify the structure of the returned analysis. It mentions 'full breakdown' but omits fields like monthly payment, interest, taxes, insurance. This is a significant gap for a tool that produces structured output, leaving the agent to guess what the tool actually returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. The 'Args' section gives brief descriptions for each parameter (e.g., 'principal: Property purchase price'). While helpful, it does not explain the calculation formula or how defaults affect output. More detail on parameter relationships would improve clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool calculates monthly mortgage payments with a full breakdown. The verb 'calculate' and resource 'mortgage payment' are specific. It naturally distinguishes from sibling tools which focus on neighborhood analysis, valuation, comparables, and listing generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'When to use' and 'When NOT to use' sections, but the guidance is generic ('structured analysis or classification') and does not explicitly compare to sibling tools. It provides some context but lacks concrete scenarios for when this tool should be chosen over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_neighborhoodA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility and delivers extensively: it declares read-only, stateless, idempotent behavior, rate limits, authentication needs, error handling, and data privacy. The dedicated 'Behavioral Transparency' section is comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections, but verbose and slightly redundant (Behavior vs. Behavioral Transparency). Could be more concise without losing key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers most aspects: purpose, usage, behavioral traits, parameter hints. However, it does not describe the output format or fields, which is a gap since there is no output schema. Nearly complete but missing return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description should compensate. It only briefly mentions location_tier values and priorities purpose in the Args line, but does not explain api_key or add detail beyond what's in the schema. Parameter semantics are minimally enhanced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes neighborhood characteristics and livability scores, with a specific verb ('analyze') and resource. It distinguishes itself from sibling tools like calculate_mortgage and estimate_valuation, which handle different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections provide clear guidance: use for structured analysis, avoid for real-time decisions without human review. This effectively differentiates appropriate contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_valuationA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and excels by detailing read-only, stateless, idempotent behavior, rate limits (10/day free), authentication requirements, error handling, and data privacy. This is far beyond minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Behavior, When to use/not, Behavioral Transparency) and front-loaded with purpose. However, it contains some redundancy (Behavior vs. Behavioral Transparency) and is somewhat lengthy, which slightly reduces conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers parameters and behavioral details well, it omits the output format (what the valuation result includes, e.g., estimated value, confidence range). Given no output schema exists, this is a notable gap for a tool with 10 parameters and no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage (no JSON schema descriptions), so the description's parameter list with one-line definitions (e.g., 'sqft: Interior square footage') adds critical semantics. All 10 parameters are covered, though the descriptions are brief and lack example values or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence explicitly states the tool estimates property valuation using comp-based methodology, which is a specific verb and resource. It clearly distinguishes from siblings like analyze_neighborhood, calculate_mortgage, and find_comparable_sales which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes dedicated 'When to use' and 'When NOT to use' sections, providing context for appropriate usage (e.g., structured analysis) and a notable exclusion (not for real-time decisions without human review). However, it does not explicitly compare to sibling tools to aid selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_comparable_salesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description comprehensively covers side effects (read-only, idempotent), authentication (basic vs. API key), rate limits (10/day), error handling (structured errors), and data privacy. However, the mention of API key via environment variable conflicts with the api_key parameter in the input schema, slightly reducing clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Behavior, When to use/not, Behavioral Transparency), but contains some redundancy (e.g., behavioral details repeated). It remains focused and front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given six parameters, no output schema, and no annotations, the description is exceptionally complete. It explains input semantics, behavioral traits, use cases, error handling, and rate limits, leaving no critical gaps for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description provides full meaning for all parameters: sqft, bedrooms, property_type (enumerated values), location_tier (enumerated values), max_results (range 1-10), and api_key (optional). This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds comparable recent sales for pricing analysis, with specific input parameters (sqft, bedrooms, property_type, location_tier, max_results). This distinguishes it from sibling tools like estimate_valuation and analyze_neighborhood.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections guide the agent. It advises use for structured analysis and warns against real-time decision-making without human review.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.