Validate dataset quality
validate_datasetScore dataset quality before trusting it using schema pass rate, completeness, duplicates, and bot-wall detection; use the score to decide between consuming data and retrying.
Instructions
Score a dataset before trusting it: schema pass rate (if a JSON Schema is given), field completeness, duplicate rate, and bot-wall detection. Use the score to decide between consuming the data and retry_low_quality_run.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | ||
| json_schema | No | JSON Schema each item should satisfy; omit for schema-free quality metrics | |
| sample_size | No | How many items to inspect (from the start of the dataset) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| total | Yes | ||
| quality | Yes | ||
| sampled | Yes | ||
| dataset_id | Yes |