quantization-calculator
Use when a user is planning to quantize an LLM to fit on smaller hardware. Given parameter count and precision transition, returns VRAM requirement, speedup estimate, and approximate quality delta.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| batch_size | No | Serving batch size (default 1) | |
| precision_to | No | Target precision (default int4) | |
| precision_from | No | Starting precision (default bf16) | |
| kv_cache_tokens | No | Max KV cache tokens (default 8192) | |
| params_billions | Yes | Model parameter count in billions |