Benchmark
benchmarkRuns real inference to measure time-to-first-token, generation tokens/sec, and prompt-processing speed, and compares multiple models on the same prompt, ranked by generation speed.
Instructions
HEAVY: Runs REAL inference. Measures a model on this machine with a warmup (absorbs load time) plus N streamed runs: time-to-first-token, generation tokens/sec (runtime-reported when available, else measured after the first token), and prompt-processing speed. Pass models[] to compare several models with the same prompt, ranked by generation speed.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "benchmark"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| runs | No | Measured runs per model (default 3) | |
| model | No | Model id/name to benchmark | |
| models | No | Compare several models (ranked by median generation tokens/sec) | |
| prompt | No | Prompt (default: a short fixed prompt) | |
| warmup | No | Run a 1-token warmup first so load time is excluded (default true) | |
| provider | No | Provider instance name. Omit to let the server choose (local providers only, unless LOCAL_AI_ALLOW_HOSTED_FALLBACK is set). | |
| maxTokens | No | Tokens to generate per run (default 128) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| results | Yes |