estimate_vram
Calculate VRAM needed to run a model at specified quantization (q4, q5, q8, fp16) and get recommended GPU tiers.
Instructions
Estimate VRAM required to run a model at a given quantization.
Args: model_id: the model ID quant: quantization level — one of 'q4', 'q5', 'q8', 'fp16'. Default 'q4'.
Returns: dict with vram_gb (estimate including context overhead) + recommended GPU tiers
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| quant | No | q4 | |
| model_id | Yes |