Server Quality Checklist

Tool Scores

• decision_gateA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It discloses the HALT behavior on FATAL answers, but does not explain what happens when all criteria pass, nor does it define 'FATAL' or the output format. This leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the purpose, and every clause earns its place. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple gate with no output schema, the description could be more complete by stating the pass condition and output behavior. It implies all four must be true to proceed, but does not explicitly describe the success return value or the meaning of FATAL. Available structured data is minimal, so the description needs to do more.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all four boolean parameters clearly (100% coverage). The description repeats the names but adds no extra semantic detail beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Run') and names a distinct resource ('Consensus Hardening Protocol R0 gate'), clearly listing the four evaluation criteria (solvable, scoped, valid, worth it). This distinguishes it from sibling tools like decision_adversary by framing it as a gate that halts work.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool ('on a proposed decision', 'before doing the work') and hints at the consequence of a FATAL result (stop and reframe). It does not explicitly compare to alternatives, but the timing guidance is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• contract_verificationA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states the tool returns gates and instructs the agent to run them, which implies the agent needs to execute commands. However, it does not disclose whether the tool itself executes anything, whether there are side effects, or what happens if gates fail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and every clause adds value: what the tool returns, what those returns consist of, and when/how to use them. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read tool with no output schema, the description covers the return value, the usage timing, and the required action. It could be slightly richer about default path behavior or failure semantics, but it is arguably sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, as the 'path' parameter is described as 'AGENTS.md path or project root'. The description adds no parameter details, but the schema already provides sufficient meaning, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return'), names the exact resource ('verification gates from a project's agent contract'), and elaborates what those gates are ('named checklists and shell commands'). This clearly distinguishes the tool from siblings like contract_load, which would handle contract loading, and decision_gate, which evaluates a decision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage context: run these gates and confirm success before declaring a task complete. It does not explicitly mention alternatives or exclusions, but the 'before work is handed off' phrase and sibling names make the appropriate use case evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• decision_adversaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It usefully discloses one-shot behavior, the adversarial mechanism, and the statuses returned. However, it does not explain side effects, required permissions, or whether the pass modifies any state, which leaves some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the action and protocol; the second gives a crisp usage rule. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately outlines what is returned ('findings plus a session status') and lists the possible statuses. It could be slightly richer on the response shape or effect of high_stakes, but it is sufficient for a 3-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no extra parameter-level detail beyond the schema; it mentions 'claim or proposed change' and 'high-stakes decision' but does not elaborate on the context or high_stakes parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Run') and a concrete resource ('Consensus Hardening Protocol adversarial pass against a claim or proposed change'), and it names the core output (scores, findings, session status). This clearly distinguishes the tool from siblings like decision_gate or engine_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use before locking any high-stakes decision.' It does not explicitly mention when not to use it or name alternatives, but the guidance is specific enough to orient an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_loadA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must carry the transparency burden. It communicates that this is an on-demand load operation and ties it to progressive disclosure, but it does not disclose return behavior, error/not-found handling, or explicit read-only guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, purposeful sentences with no filler. The core action, context, and when-to-use guidance are all packed efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter loader tool with no output schema, the description is adequate: it explains what is loaded, when to call it, and how it fits into the progressive disclosure flow. A small gap remains regarding what the response contains, but 'full SKILL.md body' conveys the essential outcome.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds minimal parameter-specific meaning beyond context ('named skill' and matching behavior), which aligns with the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Load the full SKILL.md body') on a specific resource ('for a named skill'). It also distinguishes itself from siblings like skills_list by describing this as the on-demand half of progressive disclosure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage condition: 'Call only when the current task matches the skill's description.' It does not explicitly name alternatives such as skills_list for listing skills, but the progressive-disclosure context implies the distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skills_listA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it returns only metadata (~100 tokens per skill) and covers scope resolution order, which is valuable behavioral context. However, no annotations are provided, and the description does not mention whether this performs a read-only operation, whether it follows symlinks, or how errors are handled if the project root is invalid. Still, for a listing tool, the scope and return-type disclosure is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loaded with the core purpose. It covers scope, return type, and the alternative tool in no more words than necessary. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only discovery tool with one optional parameter and no output schema, the description is complete enough: it states scope, return size, and the next step (skill_load). The only minor gap is lack of detail about exact directory patterns, but that is not essential for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is one parameter, projectRoot, with 100% schema description coverage ('Project root (defaults to cwd)'). The tool description adds the context that the root is used to discover relevant skill directories, but the schema already explains the parameter well. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: discover SKILL.md skills from a project root with specific scope details (project vs personal). It also distinguishes itself from the sibling tool skill_load by noting this returns metadata only, which helps the agent understand the difference between listing and loading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use this tool: to discover skills visible from a project root, and explicitly tells the agent to use skill_load for the full body. This provides clear usage guidance and distinguishes it from the sibling skill_load without ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• contract_loadA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It explicitly states the action (compile) and the output structure (mission, rules, etc.), implying a read-only operation without destructive effects. It does not detail error handling or side effects, but given the simple nature, the description adds sufficient context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core purpose and output, the second gives parameter usage. It is front-loaded, free of fluff, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one parameter, no output schema, and no annotations. The description covers the trigger (AGENTS.md), the processing (compile), and the output outline (structured contract with listed sections). It could explicitly state the return format, but the listed components imply what the contract contains, making it sufficiently complete for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (path described as 'AGENTS.md path or project root'). The description enriches this by clarifying that 'path' accepts a file or directory and defaults to the current working directory, providing practical usage details not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Compile') and resource ('AGENTS.md operating manual') to state exactly what the tool does: transforms a file into a structured agent contract with enumerated components (mission, rules, responsibilities, gates, skills, out-of-scope). This clearly distinguishes it from siblings like contract_verification or skills_list, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear invocation context: 'Pass a file path or a project directory (defaults to the current working directory).' This tells the agent how to supply input and what the default is, so it can decide when to call the tool. However, it does not explicitly mention alternatives or when not to use it, though the purpose is distinct enough to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• engine_statusA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the default mode does NOT spawn Python (a performance consideration), while probe=true will spawn/warm the subprocess. This is valuable behavioral context beyond what annotations would provide (none are provided). It clearly communicates the side effects of probe=true without any contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, concise, and front-loaded with the core purpose. Every sentence adds value: the first defines the tool, the second explains the parameter distinction and behavioral implications. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no nested objects), the description is complete. It covers the default behavior, the alternative with probe=true, and the performance implication. There's no missing information that would prevent correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the 'probe' parameter with 100% coverage, so the baseline is 3. The description adds context that probe=true issues a live ping and spawns the subprocess, but this largely reinforces what the schema says. It doesn't add new syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a health/readiness probe for the CHP decision engine, distinguishing it from sibling tools that load contracts, list skills, or make decisions. It specifies the resource (CHP decision engine subprocess) and the action (health/readiness probe), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the default behavior (cheap readiness snapshot without spawning Python) and when to use probe=true for a live ping. While it implies that probe=false is for quick checks and probe=true for when a live response is needed, it doesn't explicitly contrast with alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.