Server Quality Checklist

Tool Scores

• compare_stanceB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description discloses important behaviors: polarity reversal flagged as degraded, non-goal/baseline anchors silently ignored. This adds transparency beyond typical expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with key information. Could be more structured but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers behavior and constraints on anchors, but lacks explanation of state parameters and what 'degraded' means for output. Output schema exists but still feels incomplete for full understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It gives meaning to 'anchors' (filters to goal/baseline) but does not explain 'state_a' or 'state_b' beyond being string states, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it is a specialized compare for stance, with unique behavior (flags polarity reversal). It implies distinction from sibling tools but does not explicitly name them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description specifies that only goal/baseline anchors are checked and others are ignored, providing context for when to use. However, it lacks explicit guidance on when not to use or alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_proposalA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It transparently describes the limited keyword set, that most softened wording is missed, and that it only checks constraint/goal anchors. However, it does not explicitly state whether the tool is read-only or has side effects, though 'compare' suggests read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is several sentences long and includes specific details, but it could be more concise. It front-loads the purpose but then adds caveats that could be reorganized. Overall, it is adequate but not extremely efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity and 0% schema coverage, the description should cover operation and inputs comprehensively. It explains the special behavior but does not mention what the tool returns (output schema exists but not described), nor does it reinforce that both states are required. The description is incomplete for a 3-parameter tool with no schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds context about the 'anchors' parameter (checks constraint/goal anchors) but does not explain 'state_a' or 'state_b' at all. The description only partially covers the semantics of the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a specialized compare for proposals, distinguishing it from sibling tools like compare_stance or compare_commitment. It specifies the unique behavior: flags strength regression for a fixed keyword set and checks constraint/goal anchors. This provides a specific verb and resource with differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for proposals, but does not explicitly state when to use it versus alternatives. It mentions limitations (narrow keyword set, missed wording) but no direct guidance on when to choose this over compare_stance or compare_commitment.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full burden. It indicates a read-only, non-destructive operation (liveness check), but doesn't disclose failure behavior, latency, or other edge cases. Lacks thoroughness but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—a single sentence that clearly states purpose and output. No wasted words, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, straightforward liveness check), the description is nearly complete. The output schema exists and likely covers return values, so the description doesn't need to elaborate further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% and the description carries no parameter burden. Per scoring guidelines, 0 parameters warrants a baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs a 'liveness check' and returns specific fields (status, name, version). This verb+resource combination is unambiguous and distinct from sibling tools which are all 'compare' variants.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for verifying service health ('Liveness check'), but provides no explicit context on when to use vs alternatives, nor when not to use. For a simple health check, this is minimally adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_commitmentA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses the tool's behavior: it only checks commitment anchors, flags dropped promises, and acknowledges the limitation in distinguishing fulfilled from abandoned commitments.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences plus a caveat. It is front-loaded, providing immediate clarity about the tool's purpose and limitations without extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need not be described. However, the parameters lack explanation. For a specialized tool with a caveat, the description provides sufficient context for usage, but the missing parameter semantics slightly reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description does not explain the parameters (state_a, state_b, anchors) beyond the generic input schema. It adds no detail about their meaning, format, or usage, leaving ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a compare specialized for commitments, focusing only on commitment anchors, and introduces the DROPPED_SILENTLY flag. It effectively differentiates from sibling compare tools by specifying its scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly limits usage to commitment anchors and mentions the caveat about fulfilled vs abandoned. While it doesn't list when-not-to-use or alternatives, the context and sibling names imply the specialization.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compareA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: directional violation detection (only missing from state_b triggers), verbatim case-insensitive substring matching, return of 8-value status (only 4 emitted), and heuristic nature. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but slightly verbose (5 sentences). Each sentence adds value, but some pruning (e.g., combining anchor rules) could improve conciseness while maintaining clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (anchor tracking, directional violations, heuristic nature), the description covers key aspects. Output schema exists but is not shown; however, the description mentions return values sufficiently. The remaining 4 status values are not explained, but this is likely covered by output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description adds rich meaning: state_a is earlier, state_b later; anchors track specific items with detailed matching rules and violation semantics. This fully compensates for the schema's lack of parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Surface drift between two text states of the SAME decision object.' It distinguishes from sibling tools like compare_stance and compare_commitment by focusing on text state comparison and anchor tracking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use (detecting drift between same object) and how anchors work. Implicitly advises not to treat as sole gate via 'Heuristic, not a semantic judge; a drift signal, not a sole gate.' However, no explicit when-not-to-use or alternative recommendations beyond siblings listed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.