Server Quality Checklist

Tool Scores

• axe_auditB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only mentions what inputs are supported. It doesn't disclose behavioral aspects like whether this is a read-only operation, potential performance impact, error handling, authentication needs, rate limits, or what the output format looks like. The description is minimal for a tool that performs automated testing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in a single sentence that communicates the core functionality. It's appropriately sized for a tool with good schema documentation, though it could be slightly more informative given the lack of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a testing tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the audit returns, how results are structured, error conditions, or performance characteristics. The context signals show complexity (nested objects, 2 parameters), but the description doesn't provide enough information for an agent to understand the tool's behavior fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description adds minimal value by mentioning the types of URLs accepted, but doesn't provide additional context beyond what's in the schema. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run an accessibility audit'), technology used ('using axe-core via Playwright'), and supported input types ('URLs, local file paths, and localhost URLs'). It distinguishes from sibling tools by specifying the axe-core engine rather than Lighthouse or WAVE.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions what the tool supports but provides no guidance on when to use this tool versus the sibling tools (lighthouse_audit, wave_audit). There's no mention of comparative advantages, use cases, or prerequisites for choosing axe-core over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lighthouse_auditB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'Run an accessibility audit' which implies a read-only analysis operation, but doesn't disclose any behavioral traits such as execution time, resource requirements, error handling, output format, or whether it modifies any systems. For a tool with no annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that efficiently convey the core functionality and input support. Every word earns its place, and it's front-loaded with the primary purpose. There's no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters including nested objects, no output schema, and no annotations), the description is incomplete. It doesn't explain what the audit returns, how results are structured, potential limitations, or error conditions. For a tool that performs accessibility analysis, users need more context about output and behavior to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal parameter semantics beyond the schema. It mentions that the tool 'Supports URLs, local file paths, and localhost URLs,' which provides context for the 'url' parameter, but doesn't elaborate on the 'categories' or 'options' parameters. With 100% schema description coverage, the baseline is 3, and the description doesn't significantly enhance understanding of parameter usage or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run an accessibility audit') and the technology used ('using Lighthouse CLI'), which distinguishes it from sibling tools like axe_audit and wave_audit that likely use different auditing engines. However, it doesn't explicitly mention what specific accessibility aspects are audited or how it differs functionally from its siblings beyond the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying supported input types (URLs, local file paths, localhost URLs), which gives some context for when to use it. However, it provides no explicit guidance on when to choose this tool over alternatives like axe_audit or wave_audit, nor does it mention any prerequisites, exclusions, or performance considerations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wave_auditA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so effectively. It reveals the API key requirement, the temporary server behavior for local files, and the supported input types. However, it doesn't mention potential rate limits, error conditions, or what the audit output looks like, which would be helpful for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three sentences that each add distinct value: the core purpose, the API key requirement, and the local file handling behavior. There's zero redundancy or wasted words, and the information is front-loaded with the most important details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description provides adequate but incomplete context. It covers the purpose, prerequisites, and input handling well, but doesn't describe what the audit returns, potential limitations, or error scenarios. Given the complexity of an accessibility audit tool, more information about output format would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents both parameters thoroughly. The description adds some context about URL types and the temporary server behavior, but doesn't provide additional semantic meaning beyond what's already in the parameter descriptions. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run an accessibility audit') and resource ('using WAVE API'), distinguishing it from sibling tools (axe_audit, lighthouse_audit) by specifying the WAVE technology. It provides a complete verb+resource+technology combination that leaves no ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('Supports URLs and localhost URLs. Local files are automatically served via temporary local server'), but doesn't explicitly mention when NOT to use it or directly compare it to the sibling tools (axe_audit, lighthouse_audit). The API key requirement is mentioned but not framed as an alternative usage scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.