Cheap First benchmark
cheap_first_statsPublished benchmark figures. The main run is 143 questions: what Cheap First spent, what the premium model alone would have cost, how many Dolphin answered, and the average quality scores. Also includes the harder 97-question run. Does not call a model.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "cheap_first_stats"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| benchmark | Yes | The main benchmark, with costs rounded to cents the way the site quotes them, plus exactUsd. | |
| resultsPage | Yes | ||
| hardBenchmark | Yes | The harder benchmark graded by code, or null if it was not built. |