Sanitize an incoming untrusted text document, file content, user input, or RAG retrieval chunk against AI poisoning.

Strips Markdown XSS tracking pixels, neutralizes hidden zero-width Unicode steganography, redacts prompt injection phrases, and replaces high-entropy mathematical adversarial suffixes (GCG attacks).

Usage Guidelines:

• WHEN TO USE: Use on any individual raw text file, user-supplied prompt, single web page, or RAG chunk before ingesting it into the AI context window.

• WHEN NOT TO USE: Do NOT use for analyzing batches of documents for statistical dataset anomalies (use scan_dataset_for_anomalies instead) or verifying domain consensus across multiple news/search results (use verify_article_consensus instead).

Behavior & Side Effects:

• Replaces prompt injection patterns with [REDACTED_INJECTION_ATTEMPT].

• Replaces high-entropy adversarial suffixes (Shannon entropy > 4.5) with [ADVERSARIAL_SUFFIX_THREAT: REDACTED_HIGH_ENTROPY_BLOCK].

• Removes ![alt](url) tracking images, <img>, and <iframe> tracking beacons.

• Appends timestamped threat events to security_audit.json in the root workspace directory.

Scan a collection of documents, training examples, or retrieved RAG items for semantic anomalies and poisoned clusters.

Uses dense sentence embeddings (all-MiniLM-L6-v2) and Isolation Forests to detect statistical outliers that diverge from expected corpus distributions (identifying backdoor triggers, data poisoning, or trojans).

Usage Guidelines:

• WHEN TO USE: Use on collections, batches, or lists of documents (RAG retrieval sets, dataset splits, multi-file contents) to identify poisoned outlier clusters.

• WHEN NOT TO USE: Do NOT use for single-document regex sanitization, prompt injection stripping, or tracking pixel removal (use sanitize_document instead), nor for domain authority auditing across web search results (use verify_article_consensus instead).

Behavior & Side Effects:

• Computes dense vector embeddings locally (100% offline, privacy-preserving).

• Fits an Isolation Forest model and calculates centroid cosine distance metrics.

• Appends timestamped anomaly entries to security_audit.json when outliers are detected.

Verify web search results or news articles to defend against Consensus Poisoning and Sybil attacks.

Audits domain Top-Level Domains (validating trusted authorities like .gov, .edu) and calculates pairwise semantic cosine similarities to detect coordinated flooding campaigns where multiple untrusted sources syndicate near-identical (similarity > 0.95) fake consensus.

Usage Guidelines:

• WHEN TO USE: Use whenever 2 or more web search results, news articles, or online references are retrieved for a breaking topic, controversial issue, or factual query to verify that apparent consensus is not an artificial Sybil campaign.

• WHEN NOT TO USE: Do NOT use for individual document text sanitization (use sanitize_document instead) or unsupervised corpus outlier detection (use scan_dataset_for_anomalies instead).

Behavior & Side Effects:

• Audits domain provenance against verified authoritative TLDs (.gov, .edu, .mil, .int).

• Computes pairwise cosine similarity matrix across article embeddings.

• Appends timestamped alerts to security_audit.json if a coordinated Sybil attack is detected.