eval_pii_detection
Scan LLM output for PII (emails, SSNs, credit cards) using regex patterns, including jurisdiction-specific sets and custom rules. Returns a compliance score and flags matches with optional redaction.
Instructions
Detect personally-identifiable information (PII) in an LLM output.
Local-first: zero API calls. Uses a regex pattern library covering emails, phone numbers, SSNs, credit cards, IBANs, IPs, addresses, and jurisdiction-specific identifiers (HIPAA MRNs, EU VAT, California bank accounts, etc).
Score 1.0 = no PII detected. Score 0.0 = PII found (the reason field lists which types matched and example substrings).
Args:
output: The LLM-generated text to scan.
jurisdiction: Which extra pattern set to include. One of
"all" (default), "gdpr", "ccpa", "pipeda",
"hipaa", or "dpdp" (India).
custom_patterns: Optional dict of {name: regex} to add to
the default library (e.g. {"employee_id": r"EMP-\d{6}"}).
redact: If True, replace matched substrings with
[REDACTED-TYPE] markers in the reason field.
Returns:
{"score": 0.0 or 1.0, "passed": bool, "reason": str, "threshold": float, "evaluator": "pii_detection"}.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| output | Yes | ||
| redact | No | ||
| jurisdiction | No | all | |
| custom_patterns | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||