data.deduplicate
Remove duplicate CSV, TSV, JSON, JSON Lines, or YAML records using selected identity fields and report the removed count.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| keep | No | first | |
| keys | No | ||
| format | Yes | Source format; use jsonl for JSON Lines or NDJSON | |
| content | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | Structured Record deduplication result | |
| meta | Yes | ||
| service | Yes | ||
| version | Yes | ||
| request_id | Yes | Unique request identifier |