update_eval_criterion
Update a custom evaluation criterion by fully replacing its name, rubric, scale, scoring type, and config settings.
Instructions
Update a custom criterion in your account with a full replace (Pro+ only, PATCH /v1/eval-criteria/:id). name + rubric + scaleMin + scaleMax are required (not a partial update — all fields are overwritten). type / config are also fully replaced (omitting them reverts to 'llm_judge' / no config). Deterministic types require config. Global defaults (account_id IS NULL) are structurally out of scope (404); other accounts' customs are 404 too. Name collision within the account = 409.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | New name (1-50 chars, starts with an alphanumeric, [A-Za-z0-9 _\-.] only) | |
| type | No | Evaluator type (defaults to 'llm_judge' when omitted). Deterministic: 'exact_match' / 'contains' / 'regex' / 'json_schema' / 'json_path' | |
| scope | No | Evaluation scope (default call). call = per call; trajectory = per trajectory (llm_judge only) | |
| config | No | Type-specific settings (not needed for llm_judge). exact_match: {expectedOutput}, contains: {substring, caseSensitive?}, regex: {pattern, flags?}, json_schema: {schema}, json_path: {path, expectedValue?}. Categorical scoring also requires config.categories (2-10 entries, worst to best). | |
| rubric | Yes | New rubric (10-2000 chars) | |
| scaleMax | Yes | New scaleMax (1-100, greater than scaleMin) | |
| scaleMin | Yes | New scaleMin (1-100, less than scaleMax) | |
| scoreType | No | Scoring type (default numeric). boolean = pass/fail; categorical requires config.categories (llm_judge only) | |
| criterionId | Yes | Target criterion id (list_eval_criteria.criteria[].id) |