ag_data_dedupe
Deduplicate JSON records by exact or key-based matching, preserving first occurrence and order. Optionally ignore case, and report removed duplicates.
Instructions
Deduplicate JSON records exactly, optionally by key subset.
Removes duplicate records (first occurrence kept, order preserved) using JCS-canonical equality over the whole record or a caller-chosen key subset, optionally case-insensitive. Reports what was removed and why.
Deterministic, fixture-verified, free for guests (rate-limited; pass your Guild api_key to use your member budget). Returns the result plus a Guild-signed provenance envelope.
payload MUST match this JSON Schema:
{"type": "object", "properties": {"records": {"type": "array", "minItems": 1, "maxItems": 5000}, "keys": {"type": "array", "items": {"type": "string"}}, "case_insensitive": {"type": "boolean"}}, "required": ["records"], "additionalProperties": false}
Output schema: {"type": "object", "properties": {"unique": {"type": "array"}, "kept": {"type": "integer"}, "removed": {"type": "integer"}, "duplicates": {"type": "array"}}, "required": ["unique", "kept", "removed", "duplicates"], "additionalProperties": false}
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| api_key | No | ||
| payload | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||