Server Quality Checklist

Tool Scores

• calc_payday_super_deadlineB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It explicitly discloses important limitations: the tool does not invent clearing-house latency, cannot confirm LCR 2026/1 transition allocation, and cannot return ON_TIME without a fund-receipt date. This is far more transparent than a typical description, though it does not address side effects or authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and information-dense; each sentence contributes either a parameter definition, a format rule, or a limitation. The main purpose is stated clearly in the first sentence. The ordering could be slightly improved, but there is no padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a domain-specific tool with 10 parameters and no annotations, the description covers required inputs and key behavioral caveats but leaves most optional flags opaque. An agent would need significant outside domain knowledge to correctly set db_interest, out_of_cycle, first_to_fund, next_standard_qe_day, or employee_id. The output schema exists, but the input side is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters. It explains qe_day, as_at, received, and remitted, and notes ISO-8601 dates and decimal strings, but 6 of 10 parameters, including sg_amount, db_interest, out_of_cycle, first_to_fund, next_standard_qe_day, and employee_id, remain effectively unexplained or only given by their schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb and resource: 'Review one contribution against payday-super-checker,' which aligns with the tool name and the statutory/ON_TIME language. It is distinguishable from the unrelated sibling tools. However, 'payday-super-checker' itself is not elaborated, so a non-domain agent is left inferring the exact computation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: assess a contribution against the payday-super deadline. It provides useful input guidance such as as_at being required and received/remitted semantics, but it does not explicitly say when to choose this tool over an alternative or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_synthetic_sbr_fixtureB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of behavioral disclosure. It usefully discloses that the output is synthetic and non-lodgment, non-statutory-advice, but it doesn't mention side effects, persistence, or what the returned fixture corresponds to. The disclaimers are valuable but limite d.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the verb and resource, and no waste. Each sentence earns its place, with the negative disclaimers adding useful boundary information in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a small tool with an output schema and defaults on two params, the description is mostly adequate for a basic call, but it leaves the meaning of parameters and the intended use scenario implicit. The unexanded acronym CTR and absence of any alternative routing guidance mean a genuinely complete context woul d need more.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. 'CTR or BAS' gives a hint abou t the possible form_ty pe values, but entity_name and revenue_or_sales receive no explanation beyond their names and defaults. The description does not meaningfull y explain the required parameter or the two optional ones.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Generate') and resource ('synthetic CTR or BAS fixture'), so the tool's core purpose is clear. It also adds 'Not a lodgment and not statutory advice', which helps distinguish it from real submission tools, though it doesn't explicitly differentiate among the listed siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for creating synthetic test fixtures and explicitly says it is not a lodgment or statutory advice, providing a useful wnen-not cue. However, it offers no explicit guidance about when to choose this tool over siblings or what workflow it fits into.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refuse_div7aA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses the key behavior—this tool refuses Division 7A calculations—and explains why, which is meaningful context beyond the tool name. It does not specify the exact error/refusal output, but an output schema is present so return shape is covered elsewhere.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the operative behavior ('Refuse Division 7A calculations') followed by the rationale. Every word earns its place; there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Because an output schema exists, return values are already documented, and the core behavior is clear. However, with no annotations and 0% parameter documentation, the description leaves the agent to infer invocation context and parameter roles; it is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not mention borrower_name, lender_entity_name, loan_principal, start_fy, or is_secured_25_year. Parameter names are somewhat self-explanatory, but the description does not compensate for the total absence of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource: it 'Refuse[s] Division 7A calculations' and gives a concrete reason, 'No reviewed engine is wired.' This is not a tautology of the tool name and clearly differentiates the tool from the sibling calculators/benchmark tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The line 'No reviewed engine is wired' implies the tool is a safe refusal path rather than a real calculation engine, but the description never explicitly states when to call it (e.g., when a Division 7A calculation is requested) or when not to, and it names no alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ato_benchmarksA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden. It adds useful behavioral context: amounts are decimal strings, industry has a controlled vocabulary, and the operation is a comparison rather than a compliance finding. However, it does not explicitly say the tool is read-only, has no side effects, or describe error/edge behavior, leaving a moderate transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tightly written sentences front-load the main purpose, then provide parameter formatting and interpretation. There is no fluff; every sentence adds value and the structure is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 13 parameters, no annotations, and an existing output schema, the description covers the essentials: what the tool does, how amount values are formatted, and where to get valid industry names. It could also state explicitly that the tool is read-only and does not modify data, but the operation is inherently a comparison, so the gap is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description compensates meaningfully: it says all amounts are decimal strings, which applies to almost all the optional bucket parameters, and that industry must be an ATO business-type name from list_ato_benchmark_industries. It avoids restating each parameter title while giving enough shared semantics to understand the optional fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a specific verb and resource: 'Compare bucket totals against ATO small-business benchmarks.' It also distinguishes itself from sibling tools by stating it is 'not a finding that anything is wrong' and points to list_ato_benchmark_industries for valid industry names, so an agent can tell this comparison tool apart from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies when to use the tool: when comparing bucket totals to ATO small-business benchmarks. It also gives concrete usage guidance for the industry parameter by referencing list_ato_benchmark_industries. It does not explicitly state when-not-to-use alternatives or list exclusion cases, so it misses the top score, but the context is not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ato_benchmark_industriesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the behavioral burden. It discloses the data source and the default behavior when year is omitted, but it does not characterize read-only status, side effects, error cases, or result handling beyond what the output schema already covers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: a one-sentence purpose followed by two short parameter-behavior notes. Every sentence adds value and the core action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has an output schema to describe return shape, and the description covers source, purpose, and parameter behavior. It could briefly point users toward get_ato_benchmarks for retrieving actual benchmark values, but that is not necessary to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does explain both parameters: search filters by name, and year is an optional benchmark year with a concrete example and fallback behavior. It could be more precise about accepted year formats, but it is sufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('List'), resource ('ATO small-business benchmark industries'), and data source ('ato-benchmark-compare'). It is clearly differentiated from the sibling get_ato_benchmarks by focusing on the industries list rather than benchmark values.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clear context is provided for both optional parameters: search filters by name, and year selects a benchmark year with a defined default when omitted. The description does not explicitly name alternatives or exclusions, but for a simple list operation the usage intent is obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.