Create test set
create_test_setCreate and organize evaluation scenarios into test sets with names, descriptions, parameters, and metadata for structured agent testing.
Instructions
Create a test set for organizing evaluation scenarios.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"display_name": "<display_name>"
},
"name": "create_test_set"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| slug | No | URL-friendly identifier (auto-generated if not provided) | |
| parameters | No | Test case parameterization (e.g., {"name": ["Alice", "Bob"]}) | |
| description | No | Description of the test set | |
| display_name | Yes | Human-readable name for the test set | |
| test_set_type | No | Type of test set: DEFAULT, SCENARIO, TRANSCRIPT, WORKFLOW | |
| test_set_metadata | No | Additional configuration metadata |