Server Quality Checklist

Tool Scores

• recent_decisionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explicitly states it returns decisions 'from the local ledger' (indicating read-only), 'newest first' (ordering), and includes 'risk and timestamps' (fields). It does not mention side effects, auth requirements, or rate limits, but for a read-only list tool, this is sufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and no wasted words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one optional parameter, no nested objects) and existence of an output schema, the description is complete. It covers purpose, fields, and parameter usage without needing to detail return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema_description_coverage is 0%, so description must compensate. It explains the 'limit' parameter fully: how many entries to return, valid range (1-500), and default (20). This adds significant meaning beyond the schema's type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'most recent guard decisions' from the 'local ledger' and specifies the data fields (allowed, blocked, approved, risk, timestamps). It distinguishes itself from siblings like assess_action and guard_action, which perform actions rather than list decisions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (assess_action, guard_action, verify_ledger). It does not mention limitations, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_ledgerA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that it reads only local files, trusts nothing remote, and returns ok plus tampering location. Verification implies read-only; would benefit from explicit side-effect statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. First sentence states purpose and proof, second adds context. Perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, return values need not be described. No parameters, scope fully explained, behavioral constraints clear. Complete for a verification tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so baseline is 4. Description appropriately doesn't add parameter info as none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'verify' and resource 'local guard ledger's hash chain', clearly stating it proves no tampering. It distinguishes from siblings which are about assessment, guarding, and recent decisions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for local integrity verification ('trust by inspection'), but does not explicitly state when not to use or provide alternatives. No comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• guard_actionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the tool's behavior: it returns proceed true/false, a 'blocked' decision requires human approval via CLI, the agent cannot self-approve, and every decision is recorded in a tamper-evident ledger. Minor omissions include potential error scenarios or side effects beyond logging.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a lead sentence summarizing purpose, then return format, workflow details, and parameter explanations. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (pre-check with approval workflow), the description covers the core flow and return structure. It lacks details on error handling (e.g., invalid action format) but the output schema likely covers return fields. The description is sufficient for correct tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description adds meaningful semantics for both parameters: 'action' is defined as the exact command/SQL/tool call to check, and 'approval_code' as the one-time code from human CLI approval. This compensates for the schema's lack of description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Check an action BEFORE you run it' and details the return value and workflow, clearly distinguishing it from siblings like assess_action (which likely evaluates risk) and recent_decisions (which lists decisions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies when to use the tool ('before you run it') and provides a step-by-step workflow for handling blocked actions and approval codes. However, it does not explicitly compare against sibling tools or mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_actionA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: it does not record or gate, returns risk level, reversibility, dangerous capabilities, and recommendation. This is comprehensive for an assessment tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two clear sentences for purpose and usage, followed by a parameter description. Every sentence adds value, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description already lists the return fields (risk, reversible, dangerous capabilities, recommendation). The description is complete for understanding the tool's input, output, and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'action' is described as 'the command/SQL/tool call to assess (required)', which adds meaning beyond the schema's simple string type. This clarifies what kind of input is expected.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Classify' and the resource 'blast radius of an action'. It explicitly distinguishes from the sibling 'guard_action' by stating 'WITHOUT recording or gating it', making the purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use this to reason about risk; use guard_action when you actually intend to run it.' This clearly advises when to use this tool versus the alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.