Server Quality Checklist

Tool Scores

• get_fix_planA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full burden. It mentions 'scan' but does not clarify if this is read-only or destructive, nor does it discuss auth, rate limits, or side effects. The behavioral information is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence packing the action and output details. It is efficient and front-loaded, though somewhat dense. Could be restructured for readability but not excessive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains what the plan contains (signals, points, templates), providing adequate context for returned data. It does not cover error handling or URL format specifics, but is largely complete for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning to the 'url' parameter beyond the schema's own description. It neither enhances nor detracts.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a site and returns a prioritized remediation plan, specifying exactly what is included (signals, points, templates). This distinguishes it from siblings like check_signal and get_aso_framework.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (after scanning, to get a plan) but does not explicitly state when to use this tool versus alternatives like scan_site or check_signal. No when-not or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_siteA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool performs 34 checks and returns a report, but does not explicitly state that it is non-destructive or safe (only implied). No mention of rate limits, timeouts, or confirmation prompts. The security warning for include_artifacts helps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose. It is somewhat lengthy due to listing all 34 check categories, but each sentence provides meaningful detail. Could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains return values (ASO Score, maturity level, verdict, per-pillar scores, evidence, recommendations) sufficiently. It could elaborate more on the report structure, but overall complete for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant value: clarifying url format, default behavior for categories, and a critical security warning for include_artifacts (untrusted content). This goes beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: scanning a website for Agent Readiness using ASO framework and returning a detailed Agent Readiness Report with a score, maturity level, and evidence. It lists the 34 check categories, distinguishing it from sibling tools like check_signal, get_aso_framework, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives. It does not mention prerequisites, context for selection, or when not to use it. Sibling tools are not referenced, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_aso_frameworkA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description indicates a read-only retrieval with no side effects. The description is adequate for a simple reference tool but does not disclose additional behavioral traits such as rate limits or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently states the tool's output and includes a source URL. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description covers the key components of the framework. Minor gaps exist (e.g., exact list of pillars), but overall information is sufficient for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter documentation is needed. The description adds value by specifying the content returned, which complements the empty input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the ASO framework reference, listing specific content like signal pillars, weights, maturity levels, certification thresholds, and scoring rubric. It includes a source URL. The purpose is distinct from siblings like check_signal or scan_site.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the framework reference is needed, but it does not explicitly state when not to use this tool or suggest alternatives. Compared to siblings, the use case is clear but lacks direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_signalA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full disclosure burden. It mentions return values (status, evidence, fix recommendation) but does not state behavioral traits like idempotency, required permissions, or side effects. The nature of a 'check' implies read-only, but not explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences that are all necessary and front-loaded: verb and resource, then usage hint, then return summary. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and no output schema, the description covers input guidance, valid IDs source, and output shape, making it complete for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by providing concrete examples of check_id values and directing users to list_checks for valid IDs, which is helpful beyond the schema's description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Run' and the resource 'one specific agent-readiness check', with examples of check IDs. It distinguishes from siblings by implying that list_checks provides valid IDs and this tool runs one specific check, contrasting with a potential bulk scan like scan_site.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using list_checks for valid check IDs, which is a useful prerequisite. However, it does not explicitly specify when to use this tool versus alternatives like scan_site or get_aso_framework, nor provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_checksA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It correctly implies a read-only list operation and specifies what information is returned, though it does not explicitly state it has no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words. It efficiently conveys the tool's purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is complete: it explains what the tool does and what it returns. No additional information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the description adds value by detailing the output fields (id, name, category, description, spec link), which is useful beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists the full catalog of checks and specifies the returned fields (id, name, category, description, spec link), distinguishing it from sibling tools like check_signal or scan_site that perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (to see available checks) but does not explicitly state when not to use it or mention alternatives. However, the context of sibling tools helps differentiate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.