Skip to main content
Glama

Estimate local model memory

modelfinder_estimate_memory
Read-onlyIdempotent

Estimate local memory needed to run an open-weight model: weight precision plus KV cache for context length and concurrency, checked against available GPU or unified memory.

Instructions

Estimate memory to run an open-weight model locally: weights at a precision plus KV cache for a context length and concurrency (standard MHA/GQA attention only), with headroom against available memory. Give all four of layers, kv_heads, head_dim and context to include the KV cache.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
bitsYesEffective bits per weight: 16, 8, about 4.5 for common 4-bit formats.
layersNonum_hidden_layers from the model config.
contextNoContext tokens per sequence actually used.
kv_bitsNoKV cache precision in bits.
head_dimNohead_dim, or hidden_size / num_attention_heads.
kv_headsNonum_key_value_heads (or num_attention_heads if absent).
params_bYesTotal stored parameters in billions (for MoE, all experts).
concurrencyNoSequences served at once.
overhead_pctNoAllowance on top, in percent.
available_gibNoGPU or unified memory available, in GiB.
response_formatNomarkdown (default) for a readable summary, json for the full machine-readable result.markdown

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
caveatsYes
total_gibYes
weights_gibYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.3.5

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safe, read-only, idempotent profile, so the description's added value is the computational model and its limits: which components are summed, the headroom concept against available memory, and the explicit MHA/GQA-only restriction. That limitation disclosure is real behavioral context not present in the annotations. It does not discuss precision/rounding or how headroom is applied, leaving a small gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first front-loads the computation and scope, the second carries the actionable parameter rule. No filler, and the parenthetical attention restriction is placed exactly where it matters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema, rich annotations, and 100% parameter descriptions, the description only needs to explain the model and its scope, which it does. The one remaining ambiguity is behavior when only some KV inputs are supplied (it implies KV cache is omitted, but does not say whether that is an error or a graceful degradation).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes beyond the field-by-field docs by explaining an inter-parameter dependency: the four KV-related fields must all be supplied together to trigger KV-cache inclusion. That relational semantics is not encoded anywhere in the schema and is the kind of guidance an agent needs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (estimate) and resource (memory to run an open-weight model locally), and breaks the computation into its components: weights at a precision plus KV cache for context length and concurrency. It is clearly distinguishable in domain terms from the sibling estimate_cost, though it never names that sibling to sharpen the contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete conditional usage rule: 'Give all four of layers, kv_heads, head_dim and context to include the KV cache,' which tells the agent how to steer the computation. It also bounds applicability with 'standard MHA/GQA attention only.' It stops short of stating when to prefer this over the cost estimator or what to do for unsupported attention variants.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.