LLM Throughput & GPU Sizing Calculator
llm_throughput_calculatorLLM Throughput & GPU Sizing Calculator — Estimate how many GPUs your LLM needs: concurrent users and target tokens per second become cluster size, monthly cloud cost, and real utilization at load.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| gpuId | Yes | ||
| gpuHourlyUsd | Yes | ||
| concurrentUsers | Yes | ||
| utilizationHeadroomPct | Yes | ||
| targetTokensPerSecPerUser | Yes | ||
| aggregateTokensPerSecPerGpu | Yes |