Server Quality Checklist

Tool Scores

• analyze_htmlB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adds useful context by stating the engine, WCAG version, and screen size, but it does not disclose output format, potential side effects, or whether data is sent externally. This partial transparency is not fully sufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one clear sentence plus a compact bracket with key settings. Every piece of information is useful and there is no fluff. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (3 params, no output schema), the description lacks crucial information about the return value or how results are delivered. It also does not clarify when to use analyze_html vs analyze_html_json. The missing output format and usage guidance make the description incomplete for an agent to reliably use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all three parameters well-documented. The description adds minimal value beyond the schema, only hinting at default engine settings in the bracket, which is somewhat redundant with the schema's 'defaults to server config' note. No meaningful additional semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs accessibility tests on raw HTML content, specifying the action and resource. It distinguishes from analyze_url by mentioning 'raw HTML content' but does not explicitly differentiate from analyze_html_json, which likely has the same input but different output format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus siblings like analyze_url or analyze_html_json. The only implied indication is 'raw HTML content,' but without explicit exclusions or alternative recommendations, the agent is left to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_urlB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does reveal meaningful configurable details via the bracket notation: default engine ('axe'), WCAG version ('2.1_AA'), and viewport size ('1280x1024'). However, it does not disclose potential side effects (e.g., network requests), error handling, or any rate limits, and the return format is only vaguely described as 'detailed violation reports'. This is useful but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence plus a compact bracket. It conveys the core purpose and key configuration details without wasted words. The front-loaded structure immediately identifies the action and resource, and the bracket adds valuable context in a structured format. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and no annotations, the description must cover more ground. It adequately explains the tool's purpose and key parameters, but it omits guidance on when to choose this tool over siblings and what the response format looks like beyond 'detailed violation reports'. This is a common gap for tools with siblings, making the description sufficient for basic invocation but not fully complete for an autonomous agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage for all three parameters, with descriptions for 'url', 'tags', and 'engine'. The tool description does not add additional parameter semantics beyond what the schema already provides. Since the schema is fully descriptive, the baseline score of 3 is appropriate; the description offers no extra value for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Run accessibility tests on a URL and return detailed violation reports.' The verb 'Run' and resource 'URL' are specific, and the bracket notation adds useful context about engine and WCAG version. However, it does not explicitly distinguish itself from the sibling tool 'analyze_url_json', which likely provides the same accessibility tests in a different output format, so it misses an opportunity for full differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention that 'analyze_html' should be used for HTML input, or that 'analyze_url_json' might be preferred for JSON output. The context is clear for basic use, but no exclusions or alternative recommendations are given, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rulesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It conveys a read-only operation ('Get information') and adds contextual specifics (Engine: axe, WCAG: 2.1_AA, Screens: 1280x1024), but it does not disclose return format, authentication needs, rate limits, or any other behavioral nuances. This is modest transparency but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence followed by a compact bracketed note. Every word contributes meaning, and there is no redundancy. It is highly concise while still conveying the tool's purpose and contextual defaults.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 optional, well-documented parameters) and has no output schema. The description provides sufficient context: it identifies the tool's purpose, engine scope, WCAG version, and screen size. It could mention what the rule information includes (e.g., rule IDs, descriptions), but given the simplicity, the description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (tags and engine) completely, with 100% coverage. The description's reference to 'specified engine' mirrors the engine parameter and adds no new semantic information. The bracketed context hints at default configuration but does not enhance understanding of the parameters themselves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get information about available accessibility rules for the specified engine.' It uses a specific verb ('Get') and resource ('available accessibility rules'), and the mention of 'engine' distinguishes it from sibling analysis tools like analyze_url and analyze_html, which process content rather than retrieve rule metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than explicitly stated. The description indicates this tool is for retrieving rule information, but does not say when to use it versus the sibling analyze tools. There are no explicit exclusions or alternative recommendations, so guidance is minimal but not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_url_jsonA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description provides useful context beyond the schema by stating default engine (axe), WCAG level (2.1_AA), and screen size (1280x1024). It also clearly notes the output is raw JSON. It does not mention side effects, auth, or rate limits, but for a non-destructive test tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus a compact bracket of settings. It is front-loaded with the core action and output format, with no redundant words. Every piece of information earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema or annotations, so the description must cover return values. It mentions 'violations in raw JSON format' but does not describe the structure of those violations or failure modes. While the basics are covered, more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented. The description adds value by specifying the default engine and test context, which enriches the meaning of the engine parameter and the URL scope beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs accessibility tests on a URL and returns violations in raw JSON format. This distinguishes it from siblings like analyze_url (likely human-readable), analyze_html (HTML input), and analyze_html_json (HTML input with JSON output).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like analyze_url or analyze_html_json. The only hint is the name and the phrase 'raw JSON format', which implicitly suggests use when JSON output is desired, but no clear when-not or alternative references are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_html_jsonA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context like the engine (axe), WCAG version (2.1_AA), and screen size (1280x1024), but it doesn't mention side effects, permission needs, or any limitations beyond the test itself. The metadata adds value but transparency is not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus a brief metadata note—no fluff, no redundancy. It is perfectly sized for the tool's purpose and immediately front-loads the key behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward accessibility test tool with full schema coverage and no output schema, the description adequately covers the input type (raw HTML), output format (raw JSON), and key configuration (engine, WCAG, screen). It could be more detailed about the structure of violations, but given the tool's simplicity, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides complete descriptions for all three parameters (100% coverage), so the description adds no extra parameter semantics. The baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: running accessibility tests on raw HTML content and returning violations in raw JSON format. This distinguishes it from siblings like analyze_url and analyze_html, which likely handle different input types or output formats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool (raw HTML input, expecting JSON output) and sets the context with engine and WCAG defaults. However, it does not explicitly name alternatives or provide exclusions, so it falls short of the highest score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.