Tabular Dataset & Text Deduplication, Whitespace & Normalization Engine
clean_tabular_datasetDeduplicates and normalizes noisy tabular text or JSON arrays by removing duplicate rows, whitespace, HTML/control noise, and standardizing keys before indexing or analysis.
Instructions
Deduplicates repetitive lines or records, normalizes Unicode whitespace, strips HTML/control noise, and canonicalizes tabular text or JSON arrays. Use when cleaning noisy scraped lists, CSV rows, or duplicate log lines before indexing or analysis; for extracting typed JSON fields from free-form prose paragraphs, use extract_structured_json_entities instead. Stateless read-only edge call (max 32 KB, 60 RPM): accepts 'data' ('normalization_mode' defaults to 'exact_match'), supports '?demo=true' or L402 (15 sats), and returns HTTP 400 if invalid.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | Raw newline-delimited text, CSV rows, or JSON array string to deduplicate and normalize. | |
| normalization_mode | No | Deduplication key sensitivity; defaults to 'case_insensitive_trim' when omitted. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| cleaned_data | Yes | Deduplicated, whitespace-normalized records. | |
| cleaned_count | Yes | Unique non-empty record count after cleanup. | |
| original_count | Yes | Line or record count prior to deduplication. | |
| duplicates_removed | Yes | Number of duplicate records eliminated. |