Server Quality Checklist

Tool Scores

• find_citingA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions closed-world, deterministic behavior, and return structure (provenance span), but does not disclose side effects, performance, or error handling. Partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundant information. The first sentence states purpose and behavior, the second adds requirements. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and no output schema, the description covers purpose and basic behavior but omits error handling, pagination details, and parameter interdependencies. Moderately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 40%, so description should compensate. It adds context about the return format but does not explain parameters like 'module' or 'limit' beyond the schema. Provides some value but insufficient for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds citing documents in local data modules using cites/considers edges, with specific verb and resource. It distinguishes from siblings by specifying local module scope and edge types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for local modules and lists prerequisites (installed module, @duckdb/node-api), but does not explicitly state when to use or not use this tool vs alternatives like search_cases or search_legislation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_legislationA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Describes methods and jurisdictions but does not disclose safety info, rate limits, or default behavior (e.g., default method is auto but not stated). Missing read-only hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Short and front-loaded with purpose. However, the dense list of methods and jurisdictions could be better structured for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, 0% schema coverage, and no output schema, the description only covers jurisdiction, method, and offset partially. Missing details on limit, format, sortBy, return values, error handling, and rate limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%. Description adds meaning by listing jurisdiction values and method names, but does not explain each method's behavior (e.g., what 'near' means) and omits boolean from the method list present in schema. Other params like limit, format, sortBy are not described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Search Australian and New Zealand legislation.' Lists jurisdictions and methods, and distinguishes from sibling tools like search_cases and search_citing_cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions using offset for pagination and lists jurisdictions and methods. However, does not explicitly state when to use this tool over alternatives or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• citeA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool writes to a local cache, assigns biblatex-compatible keys, and performs conditional HTTP requests for freshness checks. However, it does not describe error handling, output format for refresh_source, or idempotency, leaving some behavioral traits implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first sentence introducing the tool and the second detailing both actions. It is front-loaded with the purpose and organized efficiently, with no unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (16 parameters, 2 actions) and lack of output schema, the description covers the main functions well. It specifies return values for 'add' (cite key and AGLC4 string) but omits output details for 'refresh_source'. For a tool with no annotations, this is reasonably complete but could be slightly more thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (88%), so the baseline is 3. The description does not add significant meaning beyond the schema; it mainly restates the action parameter's effect. For the 16 parameters, the schema already provides adequate descriptions, so the description adds minimal extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool writes to a local citation cache with two distinct actions: 'add' for storing/updating citations and returning a cite key and AGLC4 string, and 'refresh_source' for checking source freshness via conditional HEAD. It uses specific verbs and resources, distinguishing it from sibling tools like bibliography or format_citation by focusing on cache writing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide guidance on when to use this tool versus alternatives. It does not mention exclusions or contexts where other tools (e.g., format_citation, fetch_document_text) would be more appropriate. With no annotations to supplement, this is a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_act_structureA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool returns a tree or not-found result and mentions the algorithm (walking edges) and requirements (DuckDB, installed module). However, it does not explicitly state that the tool is read-only, idempotent, or free of side effects. The transparency is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no extraneous content. It front-loads the purpose and structure, then adds essential context (offline, closed-world) and requirements. Every sentence serves a clear function, making it highly efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema), the description covers the essential purpose and constraints but lacks details on return format structure, parameter roles for module and format, and error handling beyond a typed not-found. With no output schema, more detail on the returned tree would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (act and depth have descriptions, module and format do not). The tool description adds no parameter information beyond what the schema provides. It does not explain the 'module' parameter's purpose or the effect of 'format', leaving potential ambiguity for the agent. This fails to compensate for the schema's gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the containment tree of an Act by walking specific edges, and identifies the resource (Act) and action (return tree). It distinguishes from sibling tools by specifying 'installed local data module (offline, closed-world)', setting it apart from online legislation tools like search_legislation or fetch_document_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it works offline with installed modules and returns a tree structure. While it doesn't explicitly name alternatives, the 'offline, closed-world' phrasing implies when to use it (when local data is available) and when not (e.g., for online or real-time data). This effectively guides selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• format_citationA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that pinpoint mode fetches a judgment from AustLII, a notable side effect. However, it does not mention authentication requirements, rate limits, or error handling for network failures. With no annotations, the description carries the burden fully, but it does provide key behavioral context for a complex tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused paragraph, front-loading the tool's core purpose. Every sentence adds value: mode enumeration, requirements, and the unique AustLII functionality. It is efficient without being terse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (13 parameters, 5 modes, no output schema), the description covers the purpose of each mode and parameter interactions adequately. It lacks detail on output format or error states, but for a formatting tool this is acceptable. The description is complete enough for a legal citation expert.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is complete (100%), so the baseline is 3. The description adds semantic value by grouping parameters with modes and explaining their interactions (e.g., 'pinpointPara' and 'pinpointPage' are for short-form modes). This goes beyond the schema's individual parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a concrete verb ('Format') and resource ('Australian case citation per AGLC4 rules'), with detailed enumeration of modes (full, short, ibid, subsequent, pinpoint). This clearly distinguishes it from sibling tools like 'cite' or 'resolve_citation' which likely serve different citation systems or tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains each mode's purpose and when to use them, including required parameters for specific modes (e.g., footnoteRef for subsequent). It lacks explicit comparison to sibling tools (e.g., when to use this versus 'cite'), but the context signals and mode detail provide sufficient guidance for a legal agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_provisionA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It clearly states the tool is deterministic, offline, and resolves provisions via citable handles. It mentions dependencies and return types (provision text with provenance or typed not-found). However, it does not disclose potential side effects, rate limits, or error behavior beyond the not-found case, leaving minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of three sentences, front-loading the core purpose. It efficiently incorporates dependencies, offline nature, and fallback behavior. While clear, the structure could be slightly improved by separating prerequisites from behavior, but overall it is concise and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the return value (provision text with provenance or not-found) and mentions the fallthrough mechanism. It covers dependencies and offline context. However, it does not detail the structure of the provenance information or any performance characteristics, leaving some gaps for a complex tool with multiple parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75% (3 of 4 parameters described). The description adds limited parameter-specific meaning: it references 'citable handle' for act and provision, consistent with the schema. No extra details are provided for the module or format parameters beyond what the schema already documents. Thus, the description adds marginal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: deterministic provision lookup over installed local data modules (offline). It distinguishes from siblings by stating it does not use embedding or ranking, contrasting with semantic search tools like semantic_search_local. The mention of fallthrough to live AustLII further differentiates its scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context for when to use this tool: for offline, deterministic lookups. It explains the result type (provision text with provenance or not-found) and the fallback mechanism to live AustLII, implying when not to use it (when online or non-deterministic results are needed). Prerequisites (@duckdb/node-api and installed modules) are stated, but it does not explicitly exclude alternative tools like resolve_citation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_document_textA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description transparently discloses the conditional side effect of saving a local markdown copy and updating cache headers. It does not mention rate limits, authentication, or error behavior, but the core behavioral trait (read with potential write) is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the main purpose, and each sentence adds necessary information. No redundant or vague phrasing; it is as concise as possible while conveying conditional behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and partial parameter documentation, the description adequately covers the tool's core function and the conditional side effect. It lacks details on output format and error handling, but for a retrieval tool, the essential information is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning to the 'citeKey' parameter by explaining the conditional saving behavior. It also provides context for the 'url' parameter (AustLII). However, it does not elaborate on the 'format' parameter beyond what the schema's enum already provides. With 33% schema coverage, the description compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb ('Fetch') and resource ('full text'), targeting legislation or case URLs from a specific domain (AustLII). It also distinguishes additional behavior with citeKey, making the tool's purpose clear and distinct from siblings like 'get_provision' or 'resolve_citation'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool with and without citeKey, and the environmental condition ('AUSLAW_FETCH_SOURCES') that controls side effects. However, it does not explicitly contrast with other tools or provide 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_citationA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses validation against AustLII, fallback behavior, and mode-specific operations. However, it does not mention potential error conditions, rate limits, or required authentication, which are relevant for a tool that queries an external service.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a purpose: overall function, default behavior, and mode details. It is front-loaded with the main action and efficiently covers all necessary aspects without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (three modes), no output schema, and no annotations, the description is quite complete. It explains behavior for each mode. Minor omissions include error handling and return value specifics, but overall it provides sufficient operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%, with citation and mode having descriptions. The description adds meaning by explaining the mode logic (e.g., 'auto validates then falls back to text search'). The format parameter is only described in schema, but the tool's core modes are well-covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resolves a citation to its authoritative source using AustLII. It distinguishes three specific modes (auto, validate, search) with distinct behaviors, and the verb 'resolve' accurately describes the action. The sibling tools like 'cite' or 'format_citation' have different purposes, so this tool is well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each mode (auto for typical resolution, validate for existence check, search for text search). However, it does not explicitly state when not to use this tool compared to siblings like 'format_citation' or 'jade_lookup', which could help an agent choose more precisely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_search_localA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses gating, visible degradation when unavailable, and required dependencies. This is good, but could mention that it is read-only and does not modify data, though inferred. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and key technical details. It is efficient but somewhat dense; could be slightly more structured, but overall concise and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 params, nested object, no output schema), the description is highly complete. It covers embedding method, gating, degradation behavior, prerequisites, and even hints at format. No critical gaps for an AI agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60%, and the description adds meaning by explaining that query is embedded locally, module pinning, and that filter are pre-filters applied before ranking. This adds value beyond the schema, especially for the filter object and format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool does vector recall over installed local data modules with local embedding and cosine similarity. It distinguishes from siblings like search_cases and search_legislation by specifying 'local data modules' and the embedding process, making the purpose unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when the tool works (gated on local embedder installation and module descriptor matching) and lists prerequisites. However, it does not explicitly contrast with sibling tools or state when not to use it, missing some exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bibliographyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the read-only nature ('without network calls') and explains each operation's behavior, including the side effect of writing a .bib file for 'export'. It does not mention any potential data mutability beyond that file write.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence states the core purpose, and the second sentence enumerates the operations efficiently. It is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters and multiple operations, the description covers all operations and key parameters, including required fields for specific ops. No output schema is present, but return values are described sufficiently. It feels complete for an agent to understand usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 83%, and the description adds context beyond the schema, e.g., specifying that 'query' is required for 'op=get' and that 'outputPath' defaults to a specific path. This helps the agent understand parameter usage beyond the enum and type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads from a local citation cache without network calls, and enumerates specific operations (get, list, export, cited_by). This distinguishes it from sibling tools that involve network calls or different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use each operation (e.g., 'get retrieves one citation by cite key...', 'list (default) lists cached citations'). However, it does not explicitly state when not to use the tool or directly compare to alternatives like 'resolve_citation' or 'find_citing'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_data_modulesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explicitly states 'Reads metadata only (no DuckDB attach)', which informs the agent that this is a safe, non-destructive operation. This adds useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, each serving a clear purpose: first sentence details what is listed, second sentence provides usage guidance and behavioral note. No redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (3 optional boolean/enum parameters, no output schema), the description covers the purpose, parameters, and behavioral traits sufficiently for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all three parameters with descriptions. The description adds value by clarifying that 'includeInvalid' shows refused modules and why they didn't load, and implies the 'refresh' parameter re-scans. This enhances understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists installed local data modules with specific metadata fields (name, version, etc.). It distinguishes from sibling tools which are about search, citation, and other functions, not module introspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on using 'includeInvalid' to see refused modules, and notes it reads metadata only. However, it does not explicitly state when not to use this tool or compare with alternatives beyond the implicit distinction from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_casesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses core behaviors: searches case law, lists jurisdictions, methods, sorting, and pagination. Does not mention authentication or rate limits, but search is typically read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single informative paragraph with no wasted words. Starts with purpose, then lists options in a structured manner. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers key aspects: purpose, parameters, pagination. Could mention return format or give an example, but given no output schema, it provides enough context for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description adds significant meaning: explains each method (e.g., 'title (case names only)', 'phrase (exact match)') and how offset works for pagination. Goes beyond schema enumeration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Search' and specific resource 'Australian and New Zealand case law'. Distinguishes from sibling tools like search_citing_cases by focusing on general case law search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit allowed values for jurisdiction, method, sorting, and mentions pagination. Gives clear guidance on how to structure queries but does not explicitly differentiate from alternatives like search_citing_cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.