benchmark_create_task
Create a benchmark task for blind AI writing evaluations by specifying a category, prompt, and optional criteria. Use it to set up fair comparisons of model outputs without revealing identities.
Instructions
Create a new blind AI writing benchmark task with a category, prompt, and optional evaluation criteria.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"title": "<title>",
"category": "<category>",
"prompt": "<prompt>"
},
"name": "benchmark_create_task"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| title | Yes | Title describing the writing task | |
| prompt | Yes | The writing prompt sent identically to competing models | |
| user_id | No | User/session ID | default_user |
| category | Yes | Writing category, e.g. Emails, Technical Writing, Marketing | |
| difficulty | No | Task difficulty | medium |
| evaluation_criteria | No | Optional criteria for judging outputs |