Server Quality Checklist

Tool Scores

• check_data_freshnessA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the disclosure burden. It implies read-only operation by listing rules and explains the purpose of staleness, but it does not describe the return format or whether any side effects exist. This is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and includes a structured Args section. The additional planning law context is useful but slightly verbose; it earns its place by explaining why freshness matters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description covers purpose, usage, and parameter semantics. It does not describe the return format, but the tool's behavior is straightforward enough that this is not a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite the context signal indicating 0% schema coverage, the description's Args section explicitly defines max_age_months and its meaning ('Rules older than this many months are reported as stale'). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List rules that have not been re-verified recently') with a clear resource (rules) and condition (staleness). This clearly distinguishes it from sibling tools like get_granny_flat_rules and compare_states.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to call it ('Call this before relying on a rule for a decision') and what to do with results ('warn the user'). It does not name alternatives but clearly indicates the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_statesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does reveal an important trait: 'a state whose value is unknown is listed separately rather than omitted, so it is never silently dropped from a ranking'. This is valuable context about missing data handling. However, it does not explicitly state that the tool is read-only, nor does it describe the return format (e.g., whether it returns a table or list, ordering, etc.), leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise: two sentences with no filler. The first sentence states purpose, and the second provides usage examples plus a critical edge-case behavior. It is front-loaded with the primary action and avoids redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless tool, the description is largely complete: it states what is compared, across which scope, and includes usage examples. It also addresses the important unknown-value handling. However, since there is no output schema, the description could have explicitly described the return value structure (e.g., a ranked list or table). It implies output via 'listed separately' but does not fully specify it, leaving a minor completeness gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema description coverage is vacuously 100%. Since there are no parameters to explain, the baseline for parameter semantics is 4. The description correctly does not need to add parameter details; it focuses on the tool's behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Compare') and resource ('granny flat rules'), and explicitly scopes it to 'all eight states and territories'. This distinguishes it from sibling tools like get_granny_flat_rules, which likely targets individual states, and check_data_freshness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete example questions ('which state is easiest to build in', 'where can I build the largest granny flat') that illustrate when to use this tool. It clearly implies it's for cross-state comparisons, but it does not explicitly mention alternative tools or state when not to use it. This is clear context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_granny_flat_rulesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the Queensland exception (the state row exists only to say no standard) and that lga filters support partial matches. These are meaningful behavioral traits beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short paragraphs, front-loaded with the primary purpose, then return behavior, then arguments. Every sentence earns its place with useful information and no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description explains return content and filtering behavior, including the Queensland nuance. It lacks error/edge-case details, but for a simple lookup tool with two params, it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the Args section fully compensates: state codes are listed with examples, lga is described as optional with partial-match behavior, and concrete examples are given. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get granny flat / secondary dwelling approval rules for an Australian state.' It clarifies the return includes state-level and council-level rules, clearly distinguishing it from sibling tools about data freshness and state comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when approval rules for a state are needed, and provides optional council filtering, but it does not explicitly contrast with siblings or state when not to use. The context is clear but alternatives are not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.