Server Quality Checklist

Tool Scores

• get_fix_planA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It implies read-only scanning but doesn't explicitly state lack of side effects, required permissions, or response format limitations. Adequate but lacks explicit behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose and outputs; second sentence provides usage guidance. Concise with no unnecessary words. Could be split for readability but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 1 parameter, no output schema, and no annotations, the description explains the tool's outputs and differentiates from siblings. It provides enough context for correct invocation, though lacks details on return structure (since no output schema).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (1 parameter). Description adds minimal enrichment beyond schema: 'Website URL or domain to plan fixes for' is similar to schema description. Does not add format, validation, or usage nuances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Scan a site' and resource 'prioritized remediation plan' with specific outputs listed (signals, points, templates). Distinguishes from siblings by stating when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use this tool ('when user wants an implementation roadmap') and when to use siblings 'scan_site' and 'check_signal', providing clear decision context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_signalA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses what the tool returns (status, evidence, fix recommendation) and a default behavior (omits raw remote artifacts). While it doesn't explicitly state non-destructive nature, it is implied by 'check' and the read-only context is reasonably conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences, front-loaded with examples, and contains no redundant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately covers return structure (status, evidence, fix recommendation) and references sibling tools. It could elaborate on possible status values but is largely complete for a check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and both parameters are well-described in the schema. The description adds example values for check_id but no additional semantic meaning beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a specific agent-readiness check with examples (e.g., 'a2a-agent-card'). It explicitly distinguishes from sibling tools by contrasting use cases: targeted validation vs scan_site for complete score, get_fix_plan for roadmap, and list_checks for valid IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use (targeted validation after fix or debugging one signal) and when-not-to-use guidance, naming alternative tools (scan_site, get_fix_plan, list_checks) with their purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_siteA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavior: it runs 34 checks, returns ASO Score, maturity level, verdict, per-pillar scores, evidence, and recommendations. It also notes that include_artifacts returns untrusted data. It is thorough, though it does not explicitly state it is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of four well-structured sentences. It front-loads the core purpose, then usage guidance, then details, then output summary. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (34 checks, multiple output components) and no output schema, the description adequately lists return items. It could mention the output format but is otherwise complete for an agent to understand what the tool returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the purpose of categories (e.g., Discoverability) and the caution about include_artifacts returning untrusted data, which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: scanning a website using the ASO framework and returning an ASO Score Report. It uses specific verbs and resources, and differentiates from sibling tools like check_signal and get_fix_plan.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides when to use this tool (full baseline, competitive audit, before/after measurement) and when to use alternatives (check_signal for single signal, get_fix_plan for remediation only). This gives clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_aso_frameworkA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses that the tool is a read-only reference lookup, describing the exact content returned (pillars, weights, levels, thresholds, rubric) and confirming no side effects or destructive actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the core purpose and output, followed by usage guidance. Every sentence adds value without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description provides sufficient context for an agent to understand the tool's behavior and output. It mentions the source URL and key contents, though the exact data structure is not specified. Still, it is reasonably complete for a reference tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the description cannot add parameter-specific meaning. Baseline 4 is appropriate as there is no need for parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns the ASO framework reference, including specific components like signal pillars, weights, maturity levels, certification thresholds, and scoring rubric. This purpose is distinct from sibling tools like scan_site or check_signal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use it ('for education, documentation, or explaining how scores are calculated') and what it does not do ('it does not fetch or scan a website'), providing clear guidance relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_checksA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It transparently indicates this is a read-only catalog operation with no scanning or scoring. However, it does not explicitly mention idempotency or safety, though it is implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences: first states purpose and output, second states usage scenarios and a negation. No wasted words, well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and no annotations, the description fully covers what it does, how to use it, and what it returns. Complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (100% schema coverage). The description adds value by naming the specific fields returned, providing semantic context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists the full catalog of ASO checks with specific fields (id, name, category, description, spec link). It distinguishes itself from siblings like check_signal and scan_site by explicitly noting it does not scan a site or produce a score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises use before check_signal to discover valid check ids, to build UI filters, or to explain scanner coverage. Also states what it does not do (no scan or score), effectively guiding when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.