eval-cost-calculator
Use when a user needs to budget an LLM evaluation run. Given samples/models/trials, returns total cost, per-run cost, and parallel time estimate.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Eval model (default claude-sonnet-4-6) | |
| models | No | Candidate models (default 1) | |
| samples | Yes | Number of eval samples | |
| provider | No | Eval provider (default anthropic) | |
| avg_tokens_in | No | Avg input tokens per sample (default 2000) | |
| judge_enabled | No | Enable LLM-as-judge second pass (default false) | |
| avg_tokens_out | No | Avg output tokens per sample (default 500) | |
| judge_tokens_in | No | Judge input tokens (default 1500) | |
| judge_tokens_out | No | Judge output tokens (default 200) | |
| trials_per_sample | No | Repeats per sample (default 1) |