Server Quality Checklist

Tool Scores

• get_incidentsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only implies a read operation (retrieve) but lacks details on auth, rate limits, or other constraints. Minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. Front-loaded with the core purpose, then usage context. Excellent efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters and no output schema, the description is too brief. It doesn't explain result format, pagination, or error handling, leaving the agent with gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description adds no extra parameter meaning beyond the schema's own descriptions. No improvement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it retrieves notable Australian AI incidents, with a specific verb and resource. It distinguishes from sibling tools like search_patterns and get_regulatory_changes, which focus on patterns and regulations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides context for when to use (understanding real-world failures, enforcement actions, prevention patterns) but does not explicitly mention when not to use or compare to alternatives. Still informative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_regulatory_changesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It does not mention read-only nature, rate limits, error behavior, or data freshness. While 'retrieve' implies read-only, more explicit disclosure would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the core purpose. It is appropriately sized without redundancy, though a bit more structure (e.g., listing parameters) would improve scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three optional parameters and no output schema, the description covers the core use case but does not explain what the response looks like (e.g., list of changes with dates). This leaves some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already explains each parameter (limit, regulator, impact_level). The description adds no additional context for parameters, earning the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves 'recent and upcoming Australian AI regulatory changes' with specific regulators listed. The verb 'retrieve' and resource 'changes' are precise, and the scope distinguishes it from siblings like search_patterns or get_incidents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'Useful for understanding compliance obligations when building AI systems for the Australian market,' which indicates when to use. It does not explicitly mention when not to use or compare to siblings, but the sibling tools are clearly different in domain.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_patternsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits like safety (read-only), but it only lists return fields. It does not mention that the tool does not modify data, has no side effects, or requires no special permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no redundancy. Every sentence serves a clear purpose: stating the action and listing return fields, plus providing a use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the tool's purpose and return fields are well described. It covers the core functionality adequately for a search tool, though details like pagination or error handling are omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with clear parameter descriptions for query, category, and maturity. The tool description adds no new semantic value beyond what the schema already provides, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'search' and names the resource 'aipatterns.com.au AI pattern library', clearly distinguishing it from siblings like get_pattern (single retrieval) and other tools for different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is useful for finding patterns relevant to use cases, capabilities, or compliance, but does not explicitly tell the agent when NOT to use it or how it compares to siblings for selection. The guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_patternA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool retrieves details including 'implementation guidance and regulatory context', but does not discuss side effects, permissions, rate limits, or the exact structure of the response. For a simple read operation, this is adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first conveys purpose and content, the second provides a critical usage hint. No redundant words, and all information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description hints at the response contents (implementation guidance, regulatory context). For a simple retrieval tool with one parameter, this is sufficient but could be improved by listing expected fields or noting any pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'slug'. The description adds value by specifying 'Use the slug returned by search_patterns', which clarifies the source of the parameter and connects to a sibling tool, exceeding the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Retrieve') and resource ('full detail of a specific AI pattern'), and distinguishes from siblings by mentioning 'implementation guidance and regulatory context'. It also references the sibling tool 'search_patterns' for obtaining the slug, reinforcing the unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use the slug from 'search_patterns', indicating when to invoke this tool (after searching). It implies that this is for retrieving details, not for listing or searching. However, it does not explicitly state when not to use it or name alternatives beyond the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sector_benchmarkA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although no annotations are provided, the description discloses that this is a read operation (Get), explains what the tool returns, and notes the data source. It does not explicitly state side effects (likely none), but for a simple retrieval tool this level of detail is sufficient for an agent to understand behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two efficient sentences: the first states the core purpose, and the second enumerates the return values and data recency. Every word adds value without redundancy, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by listing return fields. It covers the important aspects (scores, rank, averages, evidence source). It could mention response format or error handling, but for a single-parameter retrieval tool the information is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'sector' has full schema coverage (100%) with its allowed values enumerated in the schema description. The tool description adds no additional meaning beyond referencing 'a specific sector', so it meets the baseline but provides no extra semantic help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the AU AI Maturity Index benchmark score for a specific sector, lists the exact return values (overall score, dimension scores, rank, national averages), and specifies the evidence base (Q2 2026). This distinguishes it from siblings focused on patterns, incidents, or regulatory changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates usage for benchmarking a sector, and the sibling tools cover distinct areas (patterns, incidents, regulatory changes), so an agent can infer when to use this tool. However, there is no explicit guidance on when not to use it or alternative tools, preventing a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.