Server Quality Checklist

Tool Scores

• get_audit_logB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations and minimal description. 'Returns' implies read-only, but doesn't disclose potential size limits, authentication needs, or what 'full' means. Important behavioral traits unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence. Efficient but could be slightly more detailed without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, description adequately states scope ('full...all compliance checks'). However, lacks output structure hints. Sibling tools provide context but description doesn't leverage them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (schema coverage 100%), so baseline is 4. Description adds no parameter info, but not needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it returns the full hash-chained audit trail of compliance checks, clearly identifying the verb and resource. It distinguishes from siblings like 'check_action' and 'set_rules' which are action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'verify_log'. Lacks context on prerequisites or scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_logA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully convey behavioral traits. It only says 'Detects any tampering' but does not disclose the tool's return value (e.g., boolean), side effects (if any), or whether it checks against a remote source. This lack of detail forces the agent to guess the behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the purpose. It is front-loaded and contains no fluff. Every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is adequate but not fully complete. It lacks details about the return value (e.g., does it return a boolean, raise an exception, or log results?). The agent needs more context to know how to handle the tool's output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the input schema is fully covered. Baseline for 0 parameters is 4, and the description adds no param info (none needed). The agent can invoke the tool without any parameter confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it verifies the integrity of the hash-chained audit log and detects tampering. The verb 'verify' and resource 'audit log' are specific, and the tool is easily distinguished from siblings like 'check_action' or 'get_audit_log'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or typical use scenarios. Without explicit instructions, an agent may not know when verification is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_actionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries full burden. It implies a read-only check with no side effects, but does not disclose auth needs, rate limits, or behavior for missing actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence with no wasted words. The purpose is front-loaded with the verb 'Check'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and no output schema, the description is mostly complete. It could benefit from mentioning the return format (e.g., boolean or status), but the core behavior is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with clear descriptions for both parameters. The description adds only an example ('e.g. delete_user'), which is marginally helpful but not necessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks whether an action is allowed or blocked by covenant rules, using specific verb 'Check' and resource 'action'. It distinguishes from siblings like set_rules and verify_log.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided. The purpose implies use for permission checking, but no alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_rulesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It states 'Set covenant rules' but does not indicate whether this is a destructive or reversible operation, what permissions are needed, or any side effects. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no unnecessary words. Each sentence contributes meaning: the first states what it does, the second gives format examples.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects, no enums, no output schema), the description covers the key aspects: purpose and parameter format. It could mention whether rules are appended or replaced, but overall it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the parameter is described in the schema. The description adds value by providing concrete syntax examples ('forbid delete_user', 'permit read_data safe to read'), which clarify the expected format beyond the generic schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Set covenant rules using permit/forbid/require syntax.' It specifies the verb 'Set' and the resource 'covenant rules', and provides example syntax. This distinguishes it from siblings like check_action, get_audit_log, and verify_log, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides syntax examples but lacks explicit guidance on when to use this tool versus alternatives (e.g., check_action, get_audit_log). It does not mention prerequisites or when not to use it. The usage context is implied but not clarified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.