Server Quality Checklist

Tool Scores

• pv_write_fileB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for disclosing behavior. It does state the conditional execution based on the ALLOW verdict, which is important, but it omits other critical behavioral details: whether the write overwrites existing content, handles directories, or returns any result. The description is too thin for a mutating operation with no annotation safety net.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the core action and then the critical constraint. There is no wasted wording, and the structure is front-loaded with the essential purpose before the gating note. It is appropriately concise without losing necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, yet the description omits how the ALLOW verdict is obtained (e.g., via pv_decide), what happens if the verdict is not ALLOW, and any post-conditions. For a write operation that depends on a prior decision, an agent lacks sufficient context to invoke it correctly in a workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both path and content, so the schema already documents parameters adequately. The description adds no extra parameter meaning beyond the gating condition, which is not parameter-specific. Per the baseline rule, a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Write content to a file' and the gating condition 'gated by PrivateVault' and 'Only happens if the verdict is ALLOW.' This is specific and distinct from the sibling tools (execute, HTTP request, decide, blocked). However, it does not explicitly name alternatives, so it slightly misses the highest bar for differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool should be used only when a PrivateVault verdict of ALLOW has been obtained, but it does not explicitly state when to use it versus siblings like pv_decide or pv_blocked. No exclusions or alternative routing are provided, leaving the agent to infer the correct usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pv_execA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description is the sole source of behavioral information. It does disclose the gating mechanism and the fact that execution is conditional on an ALLOW verdict Mend. However, it does not detail what happens on DENY, whether the command is actually executed after approval, or the response format. This leaves some uncertainty but the core behavior is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tight sentence that front-loads the core action (run a shell command) and immediately clarifies the key constraint (gated by PrivateVault). It is concise with no filler, and the two sentences are logically ordered: what it does, then the critical behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (2 parameters, no output schema), the description covers the essential nature. However, it does not specify what happens on approval (does it return output?) or on denial (error?), nor does it mention any security implications beyond the gate. With no annotations, an agent might lack clarity on expected outcomes, but the description is adequate for a basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for both parameters ('The shell command to run' and 'Working directory'), and coverage is 100%. The description adds no further parameter-level detail, such as expected format for cwd or command length limits. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: running a shell command with PrivateVault gating. It explicitly mentions the pre-execution decision step and the ALLOW condition. While it doesn't name specific sibling tools, the core functionality is clear and distinct from the file writing, HTTP request, and decision tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this is for running shell commands that require PrivateVault authorization. It implies it should be used when a decision is needed before execution, but it does not explicitly say when not to use it or mention alternatives. Sibling tools are not referenced, so an agent might not know when to pick pv_exec over pv_decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pv_blockedA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does convey that the tool is a non-mutating, agent-scoped listing of recent items. However, it does not disclose whether there are limits on 'recent', what fields are returned, or whether this covers all sessions or only the current one.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant phrasing. Every word contributes to identifying what is returned and for whom.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool the description is nearly sufficient, but it relies on undefined concepts such as 'BLOCK decisions' and an unspecified 'recent' window. It also does not explain the return structure, and there is no output schema to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is effectively 100%, so the baseline of 4 applies. There is no parameter-level ambiguity for the description to resolve.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List'), a concrete resource ('recent BLOCK decisions'), and an explicit scope ('for this agent'). It is immediately distinguishable from the sibling pv_decide because it focuses on blocked outcomes rather than decision-making or execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives such as pv_decide, or when it should be preferred over a general decision-history view. The description implies a read-only history check but does not state exclusions or competing tool conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pv_http_requestA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals the gating mechanism (PrivateVault, ALLOW-only) and the side effect of making an outbound request. However, it does not disclose return value, error handling, or what occurs if the verdict is not ALLOW. While it adds meaningful context, it lacks comprehensive behavioral detail expected without annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no superfluous content. The primary action ('Make an outbound HTTP request') is front-loaded, followed by the critical gating condition. Every word contributes to understanding the tool's function and constraint, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (3 params, no output schema, no annotations), but the description omits important details such as the return value or response format, which is especially relevant given there is no output schema. It also does not specify behavior when the verdict is not ALLOW (e.g., whether an error is returned). While the core action is covered, these gaps reduce completeness for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters (url, body, method) have descriptive text in the schema. The description does not add extra meaning to the parameters, such as formatting requirements or relationships. Baseline 3 is appropriate since the schema already documents the parameters adequately, and the description does not enrich them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: making an outbound HTTP request, with a specific gating condition (PrivateVault, only if ALLOW). It distinguishes itself from siblings like pv_exec (execution) and pv_write_file (file writing) by focusing on HTTP requests. The verb 'make' and resource 'outbound HTTP request' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the gating condition ('Only sent if the verdict is ALLOW') but does not explicitly state when to use this tool versus alternatives. It implies usage for HTTP requests but lacks guidance on when not to use it (e.g., if the request might be blocked) or how it relates to siblings. No exclusions or alternative tool mentions are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pv_decideA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'WITHOUT executing' and 'Advisory only — nothing enforces that you honor the verdict,' which are the key behavioral traits. It also hints at the structural difference in execution tools. It does not disclose the output format or error behaviors, but the core non-execution and non-enforcement aspects are clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The core action ('Ask PrivateVault for a decision WITHOUT executing') is front-loaded, and the second sentence adds essential context about advisory nature and the contrast with execution tools. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, and the schema covers parameters, but the description omits what the returned 'decision' looks like (e.g., boolean, object, status) and any error behavior. Since there is no output schema, this gap could leave an agent unsure how to interpret the result. The description adequately covers purpose and behavioral nuance, but lacks return-pointer details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters: 'args' (object) and 'capability' (string) with clear descriptions. Since schema coverage is 100%, the description adds no extra parameter guidance, which meets the baseline for a 3. It does not go beyond the schema to explain constraints, examples, or relationships between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Ask PrivateVault for a decision WITHOUT executing.' It names the resource (PrivateVault) and the action (ask for a decision) and explicitly notes the advisory-only nature, which distinguishes it from the execution-focused pv_* siblings. The contrast with 'pv_* tools where execution is structurally behind the check' further clarifies its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong context: the tool is advisory and does not execute, contrasting with sibling tools that do. This implies it should be used for pre-flight checks or when a decision is needed without side effects. However, it stops short of explicitly stating 'use this tool when you need a verdict only' or naming a specific alternative like pv_exec, so it lacks a direct when-to-use vs. when-not-to-use recommendation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.