Server Quality Checklist

Tool Scores

• calculate_scheduleA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the return format (backend JSON payload or error structure) and the calculation method example, but does not discuss side effects or permissions. Since this is a calculation tool, the lack of further disclosure is acceptable but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, uses a clear Args/Returns structure, and every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameters and gives a general return expectation. With an output schema present, it doesn't need to detail return fields. Minor gaps remain around the calculation_method options, but the schema provides a default.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero descriptions for parameters, yet the description fully explains each parameter with units and formats (e.g., annual interest rate in percent, tenure in months, calculation method example), exceeding the schema's information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves an amortization schedule for a loan configuration, using a specific verb and resource. However, it does not explicitly differentiate from sibling tools like calculate_with_prepayment, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by clearly stating its function, but provides no explicit guidance on when to use this tool versus siblings such as calculate_emi or calculate_with_prepayment.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_emiA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format ('backend JSON payload' or 'error structure') and hints at a 'backend calculation strategy', but does not explicitly state that this is a read-only operation or mention any side effects or permissions. It provides some context but not deep behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description opens with a clear one-sentence purpose, followed by a well-organized parameter list and a return statement. Every line is informative with no redundancy or filler, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description doesn't need to enumerate return fields. It covers all parameters with units, specifies the return shape, and scopes the tool to single loans. It could be slightly more explicit about the boundary with sibling tools, but it is otherwise complete for a simple calculator.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions for parameters (0% coverage). The description fully compensates by explaining each parameter: principal as loan amount, interestRate as annual percent with an example, tenure in months, and calculation_method as a backend strategy. This adds essential meaning that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'calculate' and resource 'EMI', scoped to 'a single loan scenario'. This clearly distinguishes it from sibling tools like compare_loans or calculate_with_prepayment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a single loan scenario' implies the primary use case, but the description does not explicitly list when to use this tool versus alternatives such as calculate_schedule or compare_loans. It offers no 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_with_prepaymentA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits itself. It states that the tool 'Calculate EMI outcomes' and returns 'a Backend JSON payload capturing EMI adjustments, duration changes, and savings or an error structure,' which implies a pure calculation with no side effects. However, it does not explicitly confirm that this is a read-only operation, nor does it describe edge cases, error conditions beyond 'error structure,' or how parameters like prepayment_start_month are validated. This adds some but not rich behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a concise one-sentence summary, followed by an Args list with one line per parameter, and a Returns section. Every sentence provides useful information with no filler. It is appropriately sized for a 7-parameter tool and front-loaded with the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, no annotations, no enums in schema) and an available output schema, the description covers all parameters with their semantics and describes the return structure. Minor gaps remain: it does not specify the exact valid values for prepayment_frequency and calculation_method, nor does it explain how errors are structured beyond 'or an error structure.' This is adequate but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 7 parameters. It does so by explaining each parameter's meaning, for example, 'interestRate: Annual interest rate expressed in percent (e.g. 9.5)' and 'prepayment_start_month: Month number when prepayment begins (1-indexed).' It gives examples for prepayment_frequency and calculation_method, but does not enumerate all allowed values, which limits completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Calculate EMI outcomes when regular prepayments reduce the outstanding principal' clearly states a specific verb (calculate), resource (EMI outcomes), and the special condition (prepayments). This distinguishes it from sibling tools like calculate_emi, calculate_schedule, and compare_loans, which likely don't focus on prepayment adjustments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear context for when to use the tool: 'when regular prepayments reduce the outstanding principal.' It does not explicitly name alternatives or state when not to use it, but the context is unambiguous enough for an agent to select it for prepayment scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_loansA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It states that the tool returns a 'backend JSON payload with comparison results or an error structure,' adding useful context about output and error handling. The verb 'Compare' naturally implies a non-mutating operation, which adds behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, opening with a clear purpose statement followed by Args and Returns sections. Every sentence provides value without unnecessary padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main parameters, return type, and error structure. However, it does not enumerate possible values for calculation_method (e.g., 'reducing' vs. 'flat'), and the nested scenario structure could be more detailed. Given the tool's complexity and the presence of an output schema, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for parameters, but the description compensates fully by explaining each parameter: scenarios is a list of dictionaries with required keys (name, principal, interestRate, tenure) and an optional overrides key, and calculation_method is the default strategy. This adds significant meaning beyond the minimal schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Compare EMI outputs for multiple loan scenarios in a single API call,' using a specific verb and resource. It distinguishes itself from sibling tools like calculate_emi and calculate_schedule by emphasizing the multi-scenario comparison aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when comparing multiple scenarios in one call) but does not explicitly mention when not to use it or name alternatives. Sibling tools exist, but no direct comparison or exclusion is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.