Server Quality Checklist

Tool Scores

• gov_checkA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and meets it well. It discloses fail-closed behavior ('missing tool, crash or timeout records not_run and blocks'), the side effect of writing the result back, and the operational meaning of exit_code 2 as do-not-publish. This gives the agent a clear picture of safety and side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four dense sentences with no filler: purpose and verdicts come first, then usage constraints, then return payload and error handling. Every sentence adds operational value, and critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a gate-checking tool with an output schema, the description covers return fields (decision, exit_code, missing, gates, policy_ref_ok), failure semantics, prerequisites, and the do-not-publish signal. Combined with the fully described schema and output schema, an agent has everything needed to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all four parameters including defaults and when kind override should be used. The description reinforces the manifest and evidence-sequencing context but does not add parameter-level detail beyond the schema, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Run the full holdout gate chain on a research artifact' and lists the exact verdict values (release / review_needed / block). Its lifecycle framing ('final step before publishing') distinguishes it from the related gov_init and gov_attach tools, so an agent can identify what it does without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage timing ('Use as the final step before publishing'), prerequisite ordering ('after gov_init created the project and gov_attach recorded the evidence'), and a clear don't-call condition ('Do NOT call it before evidence is attached'). It does not explicitly name gov_report as the alternative for reporting, so it stops just short of fully contrasting all siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gov_attachA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and delivers: it discloses the side effect ('decision resets to pending'), the required follow-up ('run gov_check afterwards'), merge semantics ('repeated calls merge, never drop existing entries'), the trust model ('gov_check trusts the journaled status'), and the return payload. This exceeds what any annotation set typically covers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences, zero filler, and the core purpose is front-loaded. Every sentence carries distinct information: what it does, the two usage modes, merge behavior, the trust warning, and the return value. This is dense but optimally organized for an agent scanning top-down.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 9-parameter tool with two invocation modes and an output schema, the description is complete: it covers the correct parameter combinations, the side effect, the follow-up step, and the key failure mode to avoid. The output schema handles return-value detail, so the description's brief 'Returns the updated gates...' statement is sufficient. No critical information for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds real value beyond the schema: it explains that attachments/declarations are name=value strings, and — critically — defines the two valid parameter groupings (gate+status+tool+report_ref vs review+reviewer) that the flattened schema cannot express. The per-parameter definitions remain in the schema, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Attach gate evidence, attachments, declarations or a human review to an artifact.' This is a concrete mutation action with clear objects, and it distinguishes the tool from siblings gov_init (creation), gov_check (verification), and gov_report (reporting). An agent can identify what gov_attach does without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use conditions: 'Record gate + status + tool + report_ref after a gate tool produced evidence, or review + reviewer after a human reviewed.' It also provides a when-not warning ('Do NOT claim a gate passed when no tool ran') and names gov_check as the required follow-up. However, it never contrasts with gov_init or gov_report, so routing among those siblings is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gov_initA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so well: it discloses that files are created, conditionally written only if absent, artifact.json is never overwritten, and an error is returned with no writes on conflict. It also states the return value (created files and policy_ref SHA).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: what is written, the no-overwrite guarantee, the usage workflow, and the return value. It is front-loaded with the primary action and files.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, an output schema, and full schema param coverage, the description provides enough context for correct invocation: setup workflow, safety behavior, failure condition, and return payload. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents kind, name, and directory. The description adds little beyond the schema's existing notes, though it does reinforce that directory is created if missing. This matches the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('scaffold'), resource ('holdout governance project'), and exactly what is written (policy.yml, gate-inputs.json, artifact.json). It also distinguishes itself from siblings by referencing gov_attach and gov_check in the stated workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use it once at the start of a research run and then use gov_attach and gov_check, which gives an agent clear sequencing guidance. It also states the error condition where the tool should not be used (if artifact.json already exists).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gov_reportA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the behavioral disclosure burden. It clearly states the tool is read-only, runs no gates, changes no files, and never spawns tools. This gives an agent accurate expectations about side effects and safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet information-dense. Every sentence earns its place: the first defines behavior and output, the second gives use cases, and the third names the sibling alternative and reinforces non-mutation. Key constraints are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a read-only inspection tool. It covers purpose, output shape, usage timing, alternatives, and side-effect guarantees. An output schema exists, so return-value details need not be repeated, and the sibling context makes the tool's role in the workflow clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (manifest path and optional policy path). The description adds no additional parameter-level detail beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: a read-only governance assessment of a research artifact, returned as JSON. It enumerates the exact contents (decision state, missing gates, pass/fail per gate, policy reference check) and clearly distinguishes itself from gov_check by emphasizing it does not run gates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: inspect an artifact before attaching evidence or understand why a previous check did not release. It also names the alternative (gov_check) and states the preference for fresh gate execution, plus exclusions (never mutates, never spawns tools).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.