Server Quality Checklist

Tool Scores

• behaviorlock_explainC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of behavioral disclosure. It does mention that values are 'selected' and that a reason is returned, which adds some context, but it fails to mention return format, error behavior, or whether this is a read-only operation. The description is too sparse to cover the behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, efficiently front-loaded with the verb and primary action. There is no fluff or redundant phrasing. However, it is so compact that it sacrifices necessary detail, but the structure itself is clean.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no annotations, and no output schema, yet the description covers only the basics. It does not explain what 'traces' are, the role of 'contract', or what constitutes the 'exact reason'. Given the complexity and lack of structured metadata, the description is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameters. It mentions 'baseline/candidate values', giving some meaning to those two, but 'contract' and 'assertion' are left unexplained. The phrase 'return one assertion' could confuse the 'assertion' parameter with the return value. Overall, only partial parameter insight is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares traces and returns one assertion with selected baseline/candidate values and the exact reason. It provides a specific verb ('compare') and resource ('assertion'), and the purpose is distinguishable from siblings like 'render' or 'validate', though it overlaps somewhat with 'behaviorlock_compare' which likely does the comparison without the explanation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool instead of alternatives like behaviorlock_compare or behaviorlock_validate. It implies use when an explanation is needed, but does not state this directly or list any exclusions or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• behaviorlock_renderC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does not mention side effects, return types, error behavior, or whether files are written, making the tool's runtime behavior opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly scoped sentence. Every phrase adds information about what the tool does and its supported output formats.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no annotations, and no output schema. The description is too sparse to fully contextualize its use, leaving gaps around when to invoke it and what the resulting render looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It hints that baseline and candidate are traces and lists output formats, but it does not explain the 'contract' parameter or add meaning beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action: compare traces and render in multiple formats. It differentiates from sibling tools by focusing on rendering outputs, though 'compare' overlaps somewhat with the sibling 'compare' tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like behaviorlock_compare or behaviorlock_explain. The description implies rendering use cases but does not state exclusions or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• behaviorlock_compareB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavioral traits. It mentions 'deterministic' but does not explain what the comparison output looks like, whether it has side effects, or what happens on failure. This is insufficient for a tool that produces a result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It is front-loaded and efficiently communicates the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description should explain return values or outcomes. It does not. It also lacks details about the optional contract parameter and what 'compare' means beyond a generic operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameter descriptions (0% coverage), and the description does not explain each parameter. It hints at baseline and candidate via phrasing but leaves the 'contract' parameter ambiguous and does not clarify optionality or expected formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Compare'), the resources ('baseline and candidate trace'), and the context ('against deterministic observable-behavior contracts'). It distinguishes itself from sibling tools like render, validate, explain, and fingerprint, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The sibling tool names imply different functions, but the description does not mention any conditions, exclusions, or comparisons to other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• behaviorlock_validateA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'validate' without explaining whether the tool is read-only, side-effect-free, returns success/failure, or throws errors on invalid input. This is a significant gap for a standalone validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the action and scope. There is no redundant information or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, but the description lacks usage context and behavioral transparency. It is minimally adequate for understanding the core action, but the agent needs more details about side effects, return behavior, and when to choose this tool over siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by labeling 'contract' as a behavior contract and 'traces' as optional portable trace files. However, it does not elaborate on expected formats, constraints, or how the traces relate to the contract, so it only partially bridges the gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('validate') and resource ('behavior contract and optional portable trace files'), and adds the distinguishing clause 'without comparing them' to differentiate from the sibling tool behaviorlock_compare. This makes the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without comparing them' implies a separation from comparison, but no explicit when-to-use guidance or alternatives are given. The context is implied rather than explicitly stated, so the agent must infer when to choose validate over render, explain, or fingerprint.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• behaviorlock_fingerprintA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It discloses that the operation is stable and excludes run metadata, but it does not explicitly state read-only behavior, output format, or error handling. This adds some useful context but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single, front-loaded sentence with no wasted words. It efficiently conveys purpose and key behavioral trait.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (one string input), but the description omits the meaning of the trace parameter and the return format of the fingerprint. It provides enough for basic selection but is not fully complete for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single 'trace' parameter is undocumented in the schema, and the description's phrase 'scenario status and observations' does not explicitly connect to the 'trace' argument. The description does not clarify what should be passed in the trace parameter or its format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states a specific action ('Create a stable SHA-256 fingerprint') and identifies the resource/scope ('scenario status and observations'). It distinguishes from sibling tools (render/validate/compare/explain) by its hash-generation purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for fingerprinting scenario state but does not explicitly state when to use it over siblings like behaviorlock_compare. The 'excluding run metadata' hints at a comparison use case, but no direct when-to-use guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.