Server Quality Checklist

Tool Scores

• confirm_sendA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does add a critical behavioral constraint: 'The token is single-use and time-limited.' However, it does not explain what happens after approval, whether the release is irreversible, or what errors occur with an expired/used token. This is some useful disclosure but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences contain only relevant information: the primary action and the critical token constraint. The description is front-loaded with the verb and object, has no filler, and every sentence contributes. It is appropriately concise for a simple one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter, no output schema, and no annotations, the description covers the core purpose and the most important token behavior. It is missing details like where to obtain the token, what happens if the token is invalid, and whether the release can be undone. The definition is workable but leaves several operational questions unanswered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the lone 'token' parameter. It does so by explaining that the token is single-use and time-limited, which is essential semantic meaning beyond the schema's type string. It does not specify where the token comes from or its format, but for a single simple parameter this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Approve and release'), a clear resource ('a payload'), and a precise scope ('the policy flagged for human approval'). This clearly distinguishes it from all sibling tools, which are about redacting, auditing, and LLM queries. It never resorts to tautology or restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'a payload the policy flagged for human approval' clearly indicates when to use the tool: only for payloads that are waiting on human approval. It does not explicitly enumerate alternatives or exclusions, but the sibling tools are unrelated enough that no ambiguity exists. The context is clear and enough for an agent to select it correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_audit_logA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses that the log is 'append-only' and scoped to the current session, which are useful behavioral traits. However, it does not explicitly confirm the tool is read-only, mention ordering, or describe any other operational caveats. The description is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that conveys the essential purpose and scope without any filler. Every word earns its place, and the key concept ('append-only log') leads the description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with an output schema present, the description is complete. It tells the agent exactly what is returned (log of all egress decisions this session), and the output schema covers return value details. No additional calling information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so per the baseline rule this dimension receives a 4. There is no ambiguity about arguments, and the description needs to explain nothing beyond what the input schema already makes obvious.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and resource ('append-only log of every egress decision made this session'). It clearly defines the tool's scope and distinguishes it from sibling tools like redact_preview, confirm_send, and ask_cloud_llm, which are action-oriented rather than log-retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'this session' and 'egress decision', which tells an agent this tool is for auditing session activity. However, it does not explicitly state when to use this tool versus alternatives, nor does it mention any exclusions. The usage is clear by inference, not by explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_cloud_llmA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It covers the redaction-first behavior, the policy decision, and the possible statuses (sent, blocked, needs_approval) along with the re-hydration behavior. It does not describe every edge case or auth requirement, but for a one-parameter send operation it reveals the consequential behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded, and free of filler. Every sentence contributes behavior: sending, redaction, policy decision, and status handling. The formatting of status values and the confirm_send pointer is efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, no output schema, and no annotations, the description does enough by explaining the three possible statuses and the follow-up path via confirm_send. It lacks explicit return-shape details, but that is mitigated by the status enumeration. It is slightly incomplete in not mentioning when redact_preview would be a better first step, but it remains sufficiently usable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is one required parameter, text, and schema description coverage is 0%, so the description must add meaning. It does tell the agent that the text is sent and redacted first, which gives some semantic context beyond 'Text'. However, it does not describe constraints like length, encoding, or what kind of text is acceptable, leaving room for ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Send text to the cloud LLM through the privacy gateway') and clearly distinguishes the tool from the siblings: it is the actual send path, not redact_preview, not confirm_send, and not get_audit_log. It also names the key statuses that characterize what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear situational context: the text is redacted first, then a deterministic policy decides the outcome. It points the agent to confirm_send for the needs_approval case, which helps route to a sibling. It does not explicitly discuss when redact_preview should be used or when not to call this tool, but the main alternative is at least mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redact_previewA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the safety burden and does disclose the critical non-mutating behavior: it sends nothing and issues no approval token. It also explains that it only shows the redaction and policy ruling, which is the key behavioral trait an agent needs to know. It does not discuss return structure or error cases, but for a simple preview operation the essential side-effect behavior is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with no filler; the core action is front-loaded, and the important caveat about side effects is placed at the end. Every clause contributes meaning without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one required string parameter and no output schema, the description states the input purpose, the operation, and the side-effect exclusions. It lacks explicit return structure details, but for a preview tool these are inferred from the description 'Show what would be redacted and how the egress policy would rule'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not expand on the 'text' parameter beyond implying it is the content to preview. The parameter name and type are self-explanatory, so an agent can likely use it correctly, but the description adds no detail about format, length, or interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') with a concrete resource: what would be redacted and how the egress policy would rule. It also draws a clear boundary from confirm_send by stating it does not send or issue an approval token, so an agent can tell this preview tool apart from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without sending anything or issuing an approval token' establishes when this tool should be used: as a safe preview before an actual send/approve action. It does not explicitly name alternative tools, but the context is clear enough for an agent to infer it should be called instead of confirm_send.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.