Server Quality Checklist

Tool Scores

• get_law_as_atA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the tool is known to be a safe read operation. The description adds context that the register retains every historical version and that the output shape matches get_law_text, providing useful behavioral details beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the core purpose in the first sentence and a cross-reference in the second. It is front-loaded, concise, and free of superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and no output schema, the description is incomplete. It does not explain pagination (page), section filtering (section), or the 'full' flag, nor does it define 'compilation' in detail. The reference to get_law_text hints at output structure but does not fully convey the behavior for all parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema coverage only at 20%, the description is expected to explain parameter meanings. It only hints at the 'date' parameter ('on the given date') and does not address 'titleId', 'full', 'page', or 'section'. This leaves agents without guidance for most inputs, failing to compensate for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Returns' and the resource 'Commonwealth act' with the specific temporal qualifier 'as it stood on the given date'. It also distinguishes from siblings by noting 'Same output shape as get_law_text', which signals a relationship to a likely current-version tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool—when needing a historical version on a specific date—and contrasts it with get_law_text via 'Same output shape as get_law_text'. However, it does not explicitly state when not to use it or mention alternatives like compare_versions or search_full_text.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_versionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint and openWorldHint annotations already indicating a safe read operation, the description adds valuable behavioral context by explaining the output nature (sections added/removed/reworded) and the optional line-by-line diff behavior. It does not contradict annotations and provides extra clarity beyond them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose and followed by the optional behavior. No redundant wording; every sentence contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality and optional section behavior, which is reasonably complete for a read-only diff tool with no output schema. However, it does not mention what output to expect without a section (summary vs. details) or how it relates to sibling tools like get_amendment_status, leaving slight gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 40%, with only dateA and dateB described. The description clarifies the 'section' parameter ('Give a section number for a line-by-line diff'), but titleId and page remain underspecified. It partially compensates for the coverage gap but leaves key parameter meanings implicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Shows what changed in an act between two dates' with specific detail on the types of changes (added, removed, reworded). It also distinguishes itself by specifying an optional section-level diff, which differs from sibling tools that retrieve single-version text or amendment status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for comparing versions of an act over time, but it does not explicitly state when to use this tool versus alternatives like get_law_as_at or get_amendment_status. It gives context but no exclusions or alternative guidance, leaving the agent to infer the appropriate scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_amendment_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate read-only and open-world behavior. The description further clarifies exactly what status questions the tool answers, including details about compilation and unincorporated amendments, going beyond the basic annotations. No contradiction with annotations is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence and front-loaded with the verb 'Reports'. Every listed item adds value and the sentence is free of fluff, repetition, or unrelated background.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description sufficiently covers the tool's purpose and key output dimensions for a simple one-parameter tool. However, it does not clarify the titleId input semantics or what the return structure looks like, which is a mild gap given that no output schema is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter titleId has no schema description, and the tool description does not explicitly explain titleId's value format or scope. The description's repeated use of 'title' makes the parameter's intent inferable, but the tool still relies on the parameter name for meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Reports') and names distinct outcomes: in-force status, latest compilation contents, unincorporated commenced amendments, and amending acts. This clearly identifies the tool's primary function and differentiates it from sibling tools like compare_versions or get_law_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided, nor are alternative sibling tools named. The description implies that this tool is appropriate when amendment/status information is needed, but it does not specify exclusions or preferred alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_full_textA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the description's 'slower' note adds performance context beyond the annotation. However, it does not mention potential rate limits, result size limitations, or other side effects, which might be relevant given the 'slower' warning. It is not contradictory, but additional transparency could be given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the purpose and a key comparative attribute. There is no redundancy or fluff; every word contributes to clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of annotations (read-only) and a straightforward schema, the description adequately covers the core context. It does not need to explain return values since there is no output schema, and the 'slower and broader' hint provides performance context useful for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description only explicitly covers the 'phrase' parameter ('for a phrase'), while 'limit' and 'match' are not addressed. The schema partially covers 'match' via its description, but 'limit' is left with no context beyond its name. The description adds some value by stating the search target, but it does not meaningfully enhance understanding of the other parameters, staying at a baseline level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (searches) on a specific resource (body text of every act and instrument) and includes a distinct scope (all legislation). It differentiates from siblings like search_law and get_law_text by emphasizing the full-text, cross-document nature and relevance ranking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts with the sibling tool search_law by noting it is 'slower and broader', which tells the agent when to prefer this tool over the alternative. It does not provide exhaustive usage scenarios, but the comparative hint is actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_lawA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint and openWorldHint, and the description adds value beyond that by disclosing that the tool returns title IDs and falls back to full-text matching. This provides practical operational context without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with no wasted words. It front-loads the core purpose and includes only essential additional details about return values and fallback behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward search tool with two parameters and no output schema, the description fully covers the tool's purpose, return values, and matching behavior. It leaves no critical gaps for an agent to understand when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the query parameter with an example ('income tax rates'), covering 50% of parameters. The description reinforces the 'by title' semantics but does not add significant new meaning for the limit parameter, making the guidance adequate yet not exceptional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Finds Commonwealth acts and legislative instruments on the Federal Register of Legislation by title', using a specific verb and resource. It distinguishes itself from the sibling search_full_text by noting it searches by title and falls back to full-text matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context, explaining that it returns title ids for other tools and falls back to full-text when name matching fails. However, it does not explicitly name alternatives or state conditions to avoid using this tool, so it stops short of full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_frl_healthA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety profile is covered. The description adds behavioral insight (pings, reports latency, purpose of attribution) beyond annotations. However, it doesn't specify the exact return format or what 'latency' includes, so it's slightly incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the verb 'Pings' and directly stateful. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter health check, the description communicates the tool's purpose and use case well. However, without an output schema, it would benefit from specifying what 'latency' entails (e.g., milliseconds, status indicator) to fully guide an agent interpreting results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is trivially 100%. With no parameters to document, the baseline of 4 applies; the description appropriately omits parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool pings the Federal Register of Legislation API and reports latency, which is a specific verb+resource+outcome. It distinguishes itself from sibling data-retrieval tools by focusing on API health rather than legal content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for use as an attribution/diagnostic tool ('failures elsewhere can be attributed'), implying use for health checks. It does not explicitly name alternatives or exclusions, but the sibling list makes the distinction obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_law_textA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only and open-world behavior. The description adds meaningful behavioral context: default table of sections, section-specific provision retrieval, and paginated full-text mode. It does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler, and the most important information is front-loaded. Every clause contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and well-annotated, and the description covers default behavior, section mode, and full-text mode. Minor gap: the page parameter's semantics are only implied by 'paginated' and not explicitly tied to request/response mechanics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 75% of parameters but leaves 'page' undocumented. The description explains the interplay of section and full parameters, and 'paginated' hints at the page parameter's purpose, adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Returns') and clearly identifies the resource ('the latest compilation of a Commonwealth act or instrument'). It distinguishes itself from siblings like get_law_as_at by emphasizing 'latest' and from search tools by describing structured text retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is for current text retrieval, not historical or as-at versions, by saying 'latest compilation.' However, it does not explicitly name alternative tools or state when not to use this tool, so it stops short of full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_citationsA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, so they convey safety. The description goes beyond by disclosing critical behavioral semantics: it retrieves from an open world ('openWorldHint' context), and crucially, it specifies the failure mode: 'An upstream failure is reported as UNVERIFIED, never as a missing law.' This adds essential behavioral context beyond annotations and is crucial for correct interpretation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first states the core action, the second provides a usage guideline, and the third adds a critical caveat. No wasted words, front-loaded with the verb.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (checking citation existence and status), the description covers the key behavioral expectations without going into excessive detail. It does not detail the return format or edge cases with multiple citations, but the read-only nature and the explicit UNVERIFIED rule cover the most important usage aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only one parameter, `text`, and schema description coverage is 100%, so the baseline of 3 applies. The description's mention of 'text' aligns with the parameter, but it does not add detail beyond what the schema already documents (e.g., no format, length, or examples).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource construction: 'Extracts Australian statute citations from text and checks each against the Federal Register of Legislation.' It clearly details what is checked (act existence, in-force status, section existence) and distinguishes this from siblings by focusing on verification against the register.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use before presenting legal citations to a user.' This provides clear usage context, but it does not explicitly mention alternatives or when not to use the tool, though the sibling list implies alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.