Server Quality Checklist

Tool Scores

• unmask_documentC
  2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('restore originals') without revealing side effects, whether the operation is destructive, authentication requirements, or any rate limits. The description is insufficient for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, but it is not concise because it is under-specified and in a non-English language, failing to efficiently convey necessary information. It does not earn its place as a useful guide.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters (2 required), no output schema, and no annotations, the description is critically incomplete. It does not explain return values, prerequisites, parameter behavior, or any operational context. The agent cannot determine how to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no meaning to any parameter. The parameter names (out, mode, path, vault_path) are not explained, and the enum mode (sequential/canonical) is left undefined. The description does not compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool restores originals in a previously masked file by vault, which is a specific verb+resource. However, it is in Russian, which may reduce clarity for non-Russian-speaking agents. It distinguishes from siblings like mask_document and unmask_text, but the phrase 'by vault' is ambiguous and does not fully align with the parameter name 'vault_path'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives such as mask_document or unmask_text. There is no mention of prerequisites (e.g., the file must have been previously masked) or context for when unmasking is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_textC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It reveals that the tool returns a result where 'safe_to_send=false' means the text cannot be sent, but it omits many critical details: whether the tool has side effects, what the exact output structure is, authentication requirements, or rate limits. The description is too minimal for a tool with 7 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, concise and front-loaded: first sentence states purpose, second explains a key output indicator. No redundant information. For a tool with a single core action, this is appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters and no output schema, the description is notably incomplete. It does not explain the return value format (e.g., boolean, object with 'safe_to_send'), how parameters like 'mask_urls' or 'tag_style' influence verification, or how this tool relates to the sibling 'scan_document' (both seem to check PII). An agent would lack essential context to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 86% (6 of 7 parameters have descriptions), so baseline is 3. The tool description does not add any commentary on parameters beyond what the schema already provides. For example, it doesn't explain how 'allowlist' or 'aggressive_names' affect the verification logic. The description thus provides no additional value for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the verb 'Проверить' (check) and resource 'текст' (text), clearly indicating it verifies absence of personal data. It also hints at the output semantics with 'safe_to_send=false'. This differentiates it from siblings like mask_text (which modifies) and scan_document (which likely processes documents). However, it does not explicitly specify the return type (e.g., boolean or object), slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like mask_text or scan_document. There is no mention of prerequisites, typical workflows (e.g., 'call after masking'), or exclusions. The sibling tools suggest different operations, but the description does not help an agent decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mask_documentB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains that the tool creates a copy (non-destructive), preserves formatting for certain types, and has a PDF redaction option. However, it does not describe what the 'safe of matches' is, whether it modifies the original, or any other behavioral aspects like idempotency or authorization requirements. Some transparency but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short at two sentences, with the main action in the first sentence. It avoids clutter but omits important details like the purpose of the 'safe of matches'. It is reasonably concise, though the second sentence could be more integrated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 11 parameters covering many masking options and no output schema, the description is insufficient. It explains basic file copying and format handling but fails to describe the extensive configuration options (allowlist, URL/date masking, name detection) or the structure of the 'safe of matches' output. The agent would need to rely heavily on parameter descriptions, which are incomplete (only 73% coverage).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 73%, so the baseline is 3. The description adds no extra parameter details beyond what the schema already provides. It mentions PDF redaction (pdf_redact) but doesn't elaborate on other parameters like allowlist, out, or vault_path. Thus the description does not enhance parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the primary action: creating an anonymized copy of a file and a safety mapping. It specifies different behaviors for DOCX/XLSX vs PDF, adding context about format support. However, it does not explicitly distinguish from sibling tools like mask_text or unmask_document, though the document focus is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings such as mask_text for text strings or scan_document for scanning. It lacks any conditions, prerequisites, or recommended use cases, leaving the agent to infer applicability solely from the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unmask_textB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool returns originals from a vault and describes two modes, but it does not disclose whether the operation is read-only, whether it modifies the vault, what happens on invalid vault_id or text, or any side effects. For a tool that likely interacts with a vault, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences conveying purpose and mode details. The first sentence clearly states the action, and the second elaborates on modes. It is front-loaded and efficient, though it could be slightly more structured (e.g., separating mode descriptions) without losing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and no annotations, the description is incomplete. It explains modes but does not describe the return format, prerequisites (e.g., that text must have been previously masked), error conditions, or the behavior of the text parameter. The lack of output schema information means the agent has no idea what the tool returns. This is insufficient for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% (only vault_id has a description). The description adds meaning for the mode parameter by explaining the two enum values, but it does not describe the text parameter at all. The vault_id parameter's schema description already covers its purpose. Overall, the description partially compensates for the low schema coverage by clarifying the mode parameter, but the text parameter remains undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Return originals instead of tags from the vault.' The two modes (canonical for model response, sequential for exact restoration) further specify what the tool does. The name and sibling tools (mask_text, unmask_document) help distinguish it as a text-level unmasking operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each mode (canonical for model response, sequential for exact restoration), but it does not explicitly state when to avoid this tool, nor does it compare it to alternatives like unmask_document or verify_text. The guidance is implicit through the mode descriptions, but lacks explicit exclusions or context for choosing between siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mask_textA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that it returns a vault_id for reverse substitution, but does not explain whether the operation is destructive, reversible beyond substitution, or has any side effects like logging or API rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences: the first explains what the tool does, the second gives critical timing advice. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the parameter richness (7 params, 86% schema coverage) and lack of output schema, the description adequately covers the main purpose and usage timing. It could improve by mentioning return value structure or error states, but it is sufficient for an agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 86%, so the baseline is 3. The description does not add extra meaning beyond the schema's descriptions (e.g., the allowlist, tag_style formats are already documented). The only added value is the overall workflow context, not parameter-specific details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'replace' and resource 'personal data in text', specifies the output format of stable tags like #PERSON_1#, and distinguishes itself from siblings like unmask_text and mask_document by focusing on text-level masking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this BEFORE sending someone else's text anywhere', providing clear timing guidance. However, it does not explicitly state when not to use it or describe alternative tools like verify_text or mask_document for comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_documentA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explicitly states the file is not modified and original values are not returned, which are key behavioral traits for a scan tool. It also lists supported formats, providing additional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose and key guarantees, followed by a compact list of supported formats. No extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's output nature (counts and types, no original values) but lacks a detailed return structure or error handling expectations. Given no output schema, it could be more explicit, but for a scan tool with 7 params, the provided information is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 86% (6/7 parameters have descriptions), so the schema already explains parameter meanings. The description itself does not add parameter details beyond noting file formats, which corresponds to the 'path' parameter but is already partially covered by the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Показать' - show) and resource ('что будет скрыто в файле'), clearly indicating a preview/scan function that counts and types findings. It distinguishes from siblings like mask_document (which actually modifies the file).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for non-destructive previewing but does not explicitly contrast with mask_document or other sibling tools. It provides clear context (file not modified, original values not returned) but no explicit when-to-use vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.