Server Quality Checklist

Tool Scores

• sanitize_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description adds value by stating that original identifiers are securely kept in memory. However, it omits details on read behavior, file modification, error handling, or concurrency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states the core action, and the second adds a critical security detail. Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with an output schema, the description covers the essential operation. It could mention error conditions or output format, but the presence of an output schema reduces the need. Overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add new parameter meaning beyond what is already in the schema; it only references 'selected profile' without elaboration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (reads and sanitizes), the resource (local file), and the outcome (outputs safe version for AI analysis). It distinguishes from sibling 'sanitize_text' by specifying file input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'sanitize_text' or 'reveal_text'. The description lacks context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reveal_textA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It reveals that the tool accesses a 'local volatile RAM-only session map' and performs replacement, but it does not mention authentication requirements, rate limits, side effects on the session map, or error handling for missing tokens. The description adds moderate transparency beyond schema fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 25 words with no redundant information. It is front-loaded with the core action and provides precise details efficiently. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, no nested objects, has output schema), the description covers the input, process, and data source adequately. It does not describe the output schema contents or edge cases like missing tokens, but these are not critical due to the existence of an output schema. The description is sufficiently complete for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for the sole parameter 'text' (description: 'The AI generated response containing placeholders to restore.'). The tool description adds value by specifying the format of placeholders (e.g., [EMAIL_1], [API_KEY_1]) and the source of original data (local volatile RAM-only session map), which enhances understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'replaces' and identifies the resource: masked tokens like [EMAIL_1], [API_KEY_1] in the LLM's response, using original private data from a local session map. This distinguishes it from sibling tools sanitize_file and sanitize_text, which perform the inverse operation (masking).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used after an LLM response contains masked tokens, but it does not explicitly state when to use it versus the sibling tools, nor does it provide when-not-to-use guidance or prerequisites. The usage context is clear but not formally outlined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sanitize_textA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that scrubbing is local (no data leak) and replaces matches with safe placeholders. It does not detail performance or determinism but provides sufficient behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The first sentence states the action and the second provides a practical recommendation. Front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description need not explain return values. It covers the core purpose and usage context. Could mention that output is also text with placeholders, but not necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters documented. The description adds value by listing all available profiles (e.g., 'Dev', 'Medical') and noting defaults, but this is also partly in the schema's description. Minor extra context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scrubs PII, secrets, and credentials from text and replaces them with placeholders. It specifies the action is local and differentiates from siblings like 'sanitize_file' (for files) and 'reveal_text' (presumably reverse).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using the tool before passing data to an LLM to keep data secure. It implies when to use and emphasizes 'Locally' to indicate no data is sent externally, but does not explicitly mention when not to use or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.