Create a reusable evaluation bench
tournament_create_benchSave a named benchmark of test scenarios and scoring criteria to disk, making it reusable for LLM evaluations and quick tests.
Instructions
Free, local, no model calls. Saves a new reusable bench (a named set of scenarios and scoring criteria) to disk. Once saved, it is immediately usable by tournament_evaluate and tournament_quick_test as plugin: "<name>", and it appears in tournament_list_benches. Bench names must be unique among registered benches. Draft the scenarios and criteria first and confirm them with the user before calling this tool, since saving registers the bench immediately.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Unique bench name, e.g. "customer-support-escalations". 1-60 characters, must contain at least one letter or number. Fails if a bench with this name already exists. | |
| scenarios | Yes | 1-10 scenarios that make up this bench. Each scenario is one task the candidate model is given and scored on. | |
| description | Yes | One-sentence description of what this bench tests, e.g. "Tests how well a model handles frustrated customers who feel wronged." 1-200 characters. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| file | Yes | ||
| bench | Yes |