Server Quality Checklist

Tool Scores

• detect_fileA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently states that Strac runs OCR on images/scanned documents, returns the found data element types, and does not modify the file. It does not cover edge cases like no matches or file access requirements, but the core behavior is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no fluff. It front-loads the core purpose, then gives OCR context, then states the output and non-mutating behavior. Every sentence earns its place and the length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description does not need to explain return structures. It covers the key operational aspects: supported file types, OCR behavior, data categories, and non-modification. The main gap is that it leaves the meaning of include_matched_text unexplainede and does not mention any file-size or access prerequisites, but overall it is complete enough for a detection tool with this schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate for parameter meaning. It does not mention 'path' or 'include_matched_text' at all; the optional matched-text behavior is entirely absent. The schema's property names are self-descriptive, but the description adds no value beyond the schema and fails to clarify what include_matched_text affects in the results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (detect), a resource (local file), and target data categories (PII, PHI, PCI, secrets). It also distinguishes file types and notes that OCR is applied to images/scans. However, it does not explicitly differentiate this tool from the sibling detect_sensitive_data, so the sibling advantages are not directly addressed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when you have a local file (image, PDF, scan or text) and need detection without modification. It also implies OCR handling for images/scans. But it does not explicitly state when not to use it or name alternatives such as detect_sensitive_data, and no usage exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redact_textA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the output ('Returns the redacted text plus the data element types that were found') and implies external processing via 'using Strac DLP,' but it does not mention the irreversibility of redaction, the effect of different redact_field_mode values, or whether the original text is stored or transmitted. The core behavior is clear, yet edge behaviors are left to inference.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler. The first sentence states the action, the second the return value, and the third the recommended usage context. Every sentence earns its place and the most critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for an initial selection decision, and an output schema is present to document return values. However, the lack of parameter semantics—especially include_matched_text and redact_field_mode—means an agent cannot fully understand invocation options without additional context. For a tool with no annotations, the description is not fully complete on its own.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters. It only implicitly identifies 'text' as the input, and it says nothing about redact_field_mode or include_matched_text. The parameter names and enum values offer some self-evident meaning, but an agent receives no guidance on the semantics of include_matched_text or the trade-offs between modes, which is a significant gap at this coverage level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Redact') and a precise resource ('PII, PHI, PCI and secrets out of a block of text'). It also states the return value, and the scope ('block of text') clearly distinguishes it from file-based siblings like redact_file and from detect-only siblings like detect_sensitive_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit call-to-action: 'Call this before putting untrusted or user-supplied text into a prompt, a log line, a ticket or any downstream system.' This clearly establishes when to use the tool, though it does not mention alternatives or exclusions such as 'if you only need detection, use detect_sensitive_data.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redact_fileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does disclose key behavioral facts: the original is never modified, the redacted copy is written to disk, and the path is returned. It does not cover overwrite or output-naming behavior, but the core side-effect profile is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two tight sentences, front-loads the action and scope, and each clause adds information (data types, input format, side effect, original-file safety, return value). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  It covers the main invocation context and the output, but the meaning of overwrite and output_path when null/true is left ambiguous. Since the schema provides no param descriptions and there are no annotations, an agent may not know how to set these fields correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description needed to explain path, overwrite, and output_path. It only indirectly relates to 'local file' and 'returns the path'; it never explains how overwrite or output_path behave. This leaves parameter semantics largely undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Redact') and resource ('local file'), enumerates target data types (PII, PHI, PCI, secrets) and file formats (image, PDF, scan, text), and specifies the write-to-disk behavior. This clearly differentiates it from siblings like detect_sensitive_data and redact_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly frames the tool for redacting content out of a local file, giving a clear context of use. It does not name alternatives or state exclusions, so it stops short of a 5, but the local-file qualifier makes the intended use obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_sensitive_dataA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description carries the burden. The description clearly discloses a key behavioral trait: the tool is non-mutating ('without changing it') and reports only the data element types found ('Returns the data element types Strac found'). However, it does not disclose the exact return structure, pagination, or error behavior. At least the non-destructive nature and detection-only scope are stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact three-sentence block that front-loads the core action and scope, then adds the return-value expectation and the usage-dependent sibling alternative. Every sentence contributes useful guidance with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a detection tool with only two parameters and no output schema provided, the description covers the purpose, the input domain, the output concept, and the key distinction from its sibling. The only minor gap is not clarifying the exact format of the returned element types or how include_matched_text affects the response, but given that the tool itself is conceptually simple and the return value is summarized, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema describes 'text' only as a generic string, but the description clarifies the domain and purpose of that text, namely that it contains potentially sensitive data and the tool returns the types found. For include_matched_text, there is no schema description at all, and the main description does not explicitly explain it either, though the boolean default is self-explanatory. Overall, the description adds meaningful semantic context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a clear verb-object pair, 'Detect sensitive data in text without changing it', and enumerates the exact categories (PII, PHI, PCI, credentials) the tool detects. It also explicitly contrasts the tool with redact_text, which distinguishes it from the sibling tool. The title alone would not convey this scope, but the description fully specifies the tool's unique function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states both when to use this tool — 'to decide whether text is safe to send onward' — and when to use the alternative, redact_text, 'when you need the sanitised text itself.' This explicit when/when-not guidance with a named sibling alternative is exactly what an agent needs for correct selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detokenizeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It discloses the authorization requirement, the key requirement, the live-mode restriction, and the 10-token-per-call limit. This is meaningful beyond the schema, though it doesn't cover error behavior or non-live-mode differences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, followed by critical constraints. No filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema already covers return values, so the description's focus on authorization, key requirements, and input limits is sufficient for a single-parameter tool. Minor gaps such as non-live-mode behavior are acceptable given the explicit live-mode constraint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for token_ids (0% coverage). The description compensates by defining the token format ('tkn_…') and the maximum array size (10 tokens per call), giving the agent enough to construct valid input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Resolve') and resource ('Strac vault tokens back to their original values'), and even identifies the token format ('tkn_…'). This clearly distinguishes it from the sibling tool redact_text, which presumably transforms rather than reveals data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit conditions for use: only when the caller is authorised to see raw sensitive data, and requires an IP-allowlisted server-to-server key in live mode. It doesn't explicitly mention when to use redact_text instead, but the authorization condition serves as a strong gate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.