Check answers against the answer key
check_answersRead-onlyIdempotent
The dataset's ground truth: every number it is held to, and the business's standard numbers (MRR, churn, billed
revenue, win rate...), by month, worked out when it was made. Use it to test whatever answers questions about the
data -- an agent, a dashboard, a cube, a text-to-SQL model -- and to say exactly why a wrong answer is wrong.
Call it four ways:
1. dataset_id alone: the key's numbers (`kpis`: id, title, unit, months) and `questions` to ask, drawn from
the key ("What is MRR at each month's end for 2025-07?"), each with an id. Their answers are not shown.
2. answers={question_id: number}: each answer graded within its tolerance; a wrong one is named as the
usual misreading it equals (voided invoices counted, churn over every account), when it is one.
3. values with kpi: numbers computed anywhere (ClickHouse, a semantic layer, a BI tile), as one number, a
{"2025-07": 123.4, ...} series by month, or [[month, value], ...]. Says whether they are that KPI (a tile
is read as its latest month, the year to date, ...), and what they are instead when not.
4. sql (and kpi): a query over this dataset's own tables (DuckDB, as query_dataset), run here; a wrong one is
told which KPI it computes, or the one change after which it returns the key.
The values are plain numbers, so an answer from any engine is checked the same way; export_dataset(format="demo",
dialect="clickhouse") hands over the same data and key to load into ClickHouse.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| kpi | No | ||
| sql | No | ||
| seed | No | ||
| values | No | ||
| answers | No | ||
| questions | No | ||
| dataset_id | Yes |