runs_create
Create a run. Omit prompt_id and provide output_column to score existing outputs by grading a pre-existing dataset column instead of generating new ones.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| prompt_id | No | ||
| tag_names | No | ||
| dataset_id | No | ||
| max_tokens | No | Cap on generated tokens per row. Leave unset to use the provider client's default, which is what silently truncates long outputs and makes the judge score malformed JSON. Set it to whatever the prompt uses in production so the eval matches. | |
| metric_ids | No | ||
| judge_model | No | ||
| temperature | No | Sampling temperature for generation, 0 to 1. Leave it unset, which is the default, and no temperature is sent at all, so the model applies its own. Most current frontier models refuse the parameter outright; set it only when you are targeting a model that honours it, such as anything served locally through Ollama. A refused value is re-sent without one and the run is flagged temperature_ignored. | |
| output_column | No | Dataset column to grade when prompt_id is omitted; defaults to "actual_output". | |
| expected_column | No | Dataset column holding each row's answer key / ground truth, graded by checks with compare_to "expected" and passed to the judge; defaults to "expected_output". | |
| metric_group_id | No | Attach the metrics belonging to this metric group (its current metric_ids). Ignored when metric_ids is also given. | |
| judge_temperature | No | Sampling temperature for the judge, 0 to 1. Defaults to 0 so re-judging the same output gives the same score. Raise it only to measure judge variance on purpose; any value above 0 makes the run's scores irreproducible. |