Estimate local model memory
modelfinder_estimate_memoryEstimate local memory needed to run an open-weight model: weight precision plus KV cache for context length and concurrency, checked against available GPU or unified memory.
Instructions
Estimate memory to run an open-weight model locally: weights at a precision plus KV cache for a context length and concurrency (standard MHA/GQA attention only), with headroom against available memory. Give all four of layers, kv_heads, head_dim and context to include the KV cache.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| bits | Yes | Effective bits per weight: 16, 8, about 4.5 for common 4-bit formats. | |
| layers | No | num_hidden_layers from the model config. | |
| context | No | Context tokens per sequence actually used. | |
| kv_bits | No | KV cache precision in bits. | |
| head_dim | No | head_dim, or hidden_size / num_attention_heads. | |
| kv_heads | No | num_key_value_heads (or num_attention_heads if absent). | |
| params_b | Yes | Total stored parameters in billions (for MoE, all experts). | |
| concurrency | No | Sequences served at once. | |
| overhead_pct | No | Allowance on top, in percent. | |
| available_gib | No | GPU or unified memory available, in GiB. | |
| response_format | No | markdown (default) for a readable summary, json for the full machine-readable result. | markdown |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| caveats | Yes | ||
| total_gib | Yes | ||
| weights_gib | Yes |