Server Quality Checklist

Tool Scores

• generate_edge_testsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure burden. It discloses some output quality traits (coverage, boundary, exception handling), but it fails to mention the pro-tier payment requirement and tx_hash verification, which is a significant behavioral cost, and it overclaims 'any source code' despite the schema's restricted language/framework enums.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the verb and resource, then packs relevant quality constraints into a compact list. No wasted sentences or redundant restatements of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 5-parameter tool with no output schema and no annotations, this description is incomplete. It omits what the tool returns, how tier/tx_hash affect behavior, and what language/framework limitations apply, leaving an agent without enough context to invoke it safely and correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 even without parameter details in the description. The description adds no parameter-level guidance and its 'any source code' claim is arguably inconsistent with the language enum, but the schema itself documents every parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Synthesizes') and resource ('unit tests') and enriches it with concrete qualities: 96%+ branch coverage, boundary invariance, and exception handling. This clearly separates it from sibling tools like audit_code_defenses and get_paywall_status, which concern auditing and paywall status rather than test generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the intended use case ('for any source code') and the action (synthesize tests), so an agent can infer when to choose it. It does not explicitly name alternatives or conditions when not to use it, but the sibling tools are sufficiently distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_code_defensesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It accurately conveys the static-analysis nature (auditing and detecting) and the specific analysis dimensions. However, it does not explicitly state that the tool is non-mutating, whether it accesses external resources, or what the output format is. 'Audit' strongly implies read-only, so this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tight sentence that front-loads the primary action and lists concrete audit dimensions. There is no redundancy, and every clause adds specific value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description is reasonably complete. However, with no output schema and no annotations, it does not describe what the agent should expect as a return value, how results are structured, or whether certain languages are unsupported. This leaves an agent to discover behavior at runtime.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented ('The source code to inspect' and 'Programming language'). The description adds little beyond the schema, though it gives context for why language matters (AST parsing). Baseline 3 is appropriate for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Audits') and clearly identifies the resource (source code) and the exact dimensions being analyzed (AST cyclomatic complexity, branch depth, unhandled exception vectors). It is easily distinguishable from sibling tools like generate_edge_tests and get_paywall_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when code quality and exception-safety analysis is needed, but it does not explicitly state when to use this tool versus alternatives, nor does it provide exclusion criteria or prerequisites. No when-not guidance is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paywall_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses a read behavior via 'Retrieves' but gives no detail on response format, data freshness, authentication, or failure behavior. For a zero-parameter getter this is serviceable, though not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, front-loading the action and listing the specific data items returned. Every word contributes to understanding what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema, and the description enumerates the data domains returned. It does not specify the exact return shape, but for an unparameterized getter this is adequate. An agent can invoke the tool without missing required information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty and schema description coverage is 100%, so parameter semantics are trivially satisfied. The description correctly does not dwell on parameters because there are none. The baseline of 4 applies for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Retrieves' and names exact resources: sponsorship wallet addresses, live Web3 paywall gateway, and pro tier pricing. This distinguishes it from siblings like generate_edge_tests and audit_code_defenses. Minor ambiguity remains because 'status' is not explicitly defined, but the tool's scope is still clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternative guidance is provided; the description only states what it retrieves. The intended context is inferable: call when paywall status, sponsorship, or pricing information is needed. Siblings are unrelated, so there are no competing choices that need disambiguation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.