Server Quality Checklist

Tool Scores

• calculate_mortgageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full behavioral disclosure burden. It does mention a 'bounded schedule page' which hints at paginated output, but it does not disclose whether the operation is read-only, has side effects, requires permissions, or is subject to rate limits. 'Calculate' implies a non-destructive action but not enough transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, tightly packed, and front-loads the core intent ('Calculate one scenario'). It avoids filler and tautology, but it could be slightly more structured with clauses separating the scenario, the rate, and the pagination. Still, it is efficient and easy to read.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite an output schema existing, the description is too thin for an 8-parameter tool. It does not explain how the parameters interoperate, what product types exist, or what a 'bounded schedule page' means in practice. An agent would need to guess the semantics of many fields, making this incomplete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% parameter description coverage, so the description must compensate. It does add some meaning by hinting at 'dated provider rate' (likely rate_date) and 'bounded schedule page' (schedule_limit and schedule_offset), but it leaves six other parameters (principal_eur, term_years, product_type, fixed_rate_years, start_date) completely unexplained, offering weak semantic guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'Calculate one scenario' which is a clear, specific verb and resource. It also contrasts implicitly with the 'compare_mortgages' sibling by indicating a single scenario, but it doesn't explicitly name or contrast alternative tools, so it stops short of a full 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage gates: 'one scenario' suggests a single calculation rather than comparison, and 'using a dated provider rate' implies a prerequisite. However, it does not explicitly state when to use this tool over siblings like compare_mortgages or list_mortgage_products, nor does it provide exclusions or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_interest_ratesB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden, but it only states that the tool 'returns' data, which implies a read-only operation. It does not disclose any side effects, authorization requirements, rate limits, or behavior when the requested date has no rates available; still, it is not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant phrasing. Every word contributes to understanding the returned data, and no important structured property is repeated or contradicted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter and an output schema, the description is minimally viable, but not complete. It fails to clarify the semantic relationship between the 'as_of' parameter and the returned rates' effective dates, and it offers no clue about how this tool relates to the mortgage siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the only parameter 'as_of', and the description adds almost nothing about how this parameter changes the result. Phrases like 'effective dates' and the title 'Get dated interest rates' hint at date filtering, but the description does not explicitly say whether as_of selects rates effective on, before, or after the given date.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'Return available rates with effective dates and source metadata,' which is a specific verb plus resource and clearly describes the output. It does not explicitly reference sibling tools, but the focus on dated rates with source metadata is enough to set it apart from mortgage products and calculation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to choose this tool over siblings such as calculate_mortgage, compare_mortgages, or list_mortgage_products. It does not say what kind of request or scenario best fits this tool, nor does it mention any restrictions or prerequisites such as authorship.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_mortgagesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It states that the tool does not recommend a product, which is useful, but it does not disclose what exactly happens during comparison (e.g., returns a list of products with rates, or side-by-side metrics). It also doesn't mention any side effects or requirements, so the agent has limited understanding of the tool's behavior beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and front-loads the purpose and key qualifier. There is no fluff, and it conveys the essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (5 parameters, no annotations, no schema descriptions), the description is minimal. It explains what it does (compare) and its stance (no recommendation), but it lacks details about expected output structure, how to handle results, any assumptions about input formats, or whether it fetches live rates. With an output schema present, return format may be covered, but the description still doesn't fully equip an agent to call it correctly for all scenarios, especially when to use it vs. alternatives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for parameter meaning, but it does not. The description mentions 'equal inputs' but does not explain the parameters like principal_eur, term_years, fixed_rate_years, start_date, or rate_date. However, many parameter names are self-explanatory, and the output schema exists, which may clarify the output. Since the schema provides no property descriptions, the description's lack of parameter detail is a gap, but the names help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: comparing products with equal inputs and explicitly says it does not recommend one. This is a specific verb ('compare'), resource ('mortgage products'), and a key qualifier ('without recommending'). It distinguishes itself from single-product calculation tools, though it does not name the sibling tool for that, but the context is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear condition for use: 'using equal inputs' and 'without recommending one,' which implies it is for objective comparison. However, it does not explicitly state when NOT to use this tool versus alternatives like 'calculate_mortgage' or 'list_mortgage_products.' It provides some usage context but lacks explicit exclusions or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that the tool returns health status, with no mention of side effects, permissions, rate limits, or whether it is a read-only operation. The description does not add any behavioral context beyond the bare purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the core purpose. There is no extraneous information, and every word earns its place. It is highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and an output schema present (as indicated by context signals), the description is adequately complete. It tells the agent what the tool does, and the output schema presumably details the structure of the health status response. The only minor gap is the lack of explicit mention that this is a read-only operation, but that is a small omission given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to elaborate on parameter semantics since there are none, and the schema is fully covered (100%). No additional meaning is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Return the service health status.' It is unambiguous and immediately distinguishable from the sibling tools, which all concern mortgage calculations and products. The title and description align perfectly with the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for checking server health but does not explicitly state when to use it versus alternatives or any exclusions. Given the sibling tools are all mortgage-related, the context makes the intended use obvious, but the description offers no explicit guidance on scenarios where this tool is the right choice or when it is not.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mortgage_productsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'List' implies a read-only operation and 'configured provider' hints at an external dependency, but it does not disclose behaviors like failure modes when the provider is unconfigured or whether results are static or dynamic. This is adequate but minimal for a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that directly states the action and object without filler. It is front-loaded and every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is functionally complete: it tells the agent what the tool does. It falls slightly short by not elaborating on the nature of 'constraints', but the presence of an output schema reduces the need to describe return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description adds no parameter-level detail, and none is needed. The baseline for a 0-parameter tool is 4, as there is nothing to document beyond the schema's empty properties object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('products and constraints') sourced from the configured provider. This distinguishes it from siblings like calculate_mortgage, compare_mortgages, and get_interest_rates, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus the sibling tools, nor are there any prerequisites or notes about provider configuration. The agent must infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.