Skip to main content
Glama
Sign In

MakerPortal compute

by ai.makerportal
OverviewDiscussionsAdmin

Decide whether one LLM fits one accelerator, at every quantization.

llmVramFit

Decide whether one LLM fits one accelerator, at every quantization.

The identical function behind the /lab/llm-vram instrument. Returns one row per quantization (FP16 through Q3_K_M) with weight bytes and their basis, headroom, the largest context that fits, and the bandwidth-limited decode ceiling; plus the chosen best-fitting quant, the full-precision row, and a five-state verdict. Model geometry comes from each repo’s own config.json and tensor-shape index; accelerator capacity and bandwidth come from the site’s device table.

Every result carries provenance.canonicalUrl — the published page for these exact inputs, or the lane hub when they are off the published grid.

Input Schema

NameRequiredDescriptionDefault
gpuYesAccelerator slug.
batchNoConcurrent sequences sharing the cache. Omit it for the published page's 1.
modelYesModel slug.
contextNoContext length in tokens used for the headline verdict. Omit it and the solver takes its own default of 8192, which is what the published page states.
kvBytesNoBytes per KV cache element: 2 for FP16/BF16 cache, 1 for an 8-bit cache. Omit it for the published page's 2.

Output Schema

NameRequiredDescriptionDefault
inputsYesThe parsed inputs, echoed back with defaults filled in.
resultYesThe solved analysis. Shape is per-tool.
licenseYes
provenanceYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / model / enum
      Previous value: -[
-  "deepseek-r1-distill-llama-70b",
-  "deepseek-r1-distill-llama-8b",
-  "deepseek-r1-distill-qwen-1-5b",
-  "deepseek-r1-distill-qwen-14b",
-  "deepseek-r1-distill-qwen-32b",
-  "deepseek-r1-distill-qwen-7b",
-  "dolphin-mistral-24b-venice-edition",
-  "falcon3-7b-instruct",
-  "gemma-2-27b-it",
-  "gemma-2-9b-it",
-  "gemma-3-12b-it",
-  "gemma-3-1b-it",
-  "gemma-3-27b-it",
-  "gemma-3-4b-it",
-  "gpt-oss-120b",
-  "gpt-oss-20b",
-  "granite-3-3-8b-instruct",
-  "hermes-4-3-36b",
-  "hy3",
-  "kat-coder-v2-5-dev",
-  "laguna-s-2-1",
-  "laguna-xs-2-1",
-  "llama-3-1-70b-instruct",
-  "llama-3-1-8b-instruct",
-  "llama-3-1-nemotron-70b-instruct",
-  "llama-3-2-1b-instruct",
-  "llama-3-2-3b-instruct",
-  "llama-3-3-70b-instruct",
-  "minicpm5-1b",
-  "mistral-7b-instruct-v0-3",
-  "mistral-nemo-instruct-2407",
-  "mistral-small-24b-instruct-2501",
-  "mixtral-8x7b-instruct-v0-1",
-  "nanbeige4-2-3b",
-  "olmo-2-1124-13b-instruct",
-  "phi-3-5-mini-instruct",
-  "phi-4",
-  "phi-4-mini-instruct",
-  "qwen-agentworld-35b-a3b",
-  "qwen2-5-0-5b-instruct",
-  "qwen2-5-1-5b-instruct",
-  "qwen2-5-14b-instruct",
-  "qwen2-5-32b-instruct",
-  "qwen2-5-3b-instruct",
-  "qwen2-5-72b-instruct",
-  "qwen2-5-7b-instruct",
-  "qwen2-5-coder-32b-instruct",
-  "qwen2-5-coder-7b-instruct",
-  "qwen3-0-6b",
-  "qwen3-1-7b",
-  "qwen3-14b",
-  "qwen3-30b-a3b",
-  "qwen3-32b",
-  "qwen3-4b",
-  "qwen3-8b",
-  "qwq-32b",
-  "qwythos-9b-claude-mythos-5-1m",
-  "smollm2-1-7b-instruct",
-  "smollm2-135m-instruct",
-  "smollm2-360m-instruct",
-  "tinyllama-1-1b-chat-v1-0",
-  "yi-1-5-34b-chat"
-]New value: +[
+  "deepseek-r1-distill-llama-70b",
+  "deepseek-r1-distill-llama-8b",
+  "deepseek-r1-distill-qwen-1-5b",
+  "deepseek-r1-distill-qwen-14b",
+  "deepseek-r1-distill-qwen-32b",
+  "deepseek-r1-distill-qwen-7b",
+  "dolphin-mistral-24b-venice-edition",
+  "falcon3-7b-instruct",
+  "gemma-2-27b-it",
+  "gemma-2-9b-it",
+  "gemma-3-12b-it",
+  "gemma-3-1b-it",
+  "gemma-3-27b-it",
+  "gemma-3-4b-it",
+  "gpt-oss-120b",
+  "gpt-oss-20b",
+  "granite-3-3-8b-instruct",
+  "hermes-4-3-36b",
+  "hy3",
+  "k2-horizon-7b",
+  "kat-coder-v2-5-dev",
+  "laguna-s-2-1",
+  "laguna-xs-2-1",
+  "llama-3-1-70b-instruct",
+  "llama-3-1-8b-instruct",
+  "llama-3-1-nemotron-70b-instruct",
+  "llama-3-2-1b-instruct",
+  "llama-3-2-3b-instruct",
+  "llama-3-3-70b-instruct",
+  "minicpm5-1b",
+  "minicpm5-2b",
+  "mistral-7b-instruct-v0-3",
+  "mistral-nemo-instruct-2407",
+  "mistral-small-24b-instruct-2501",
+  "mixtral-8x7b-instruct-v0-1",
+  "nanbeige4-2-3b",
+  "nex-n2-5-pro",
+  "olmo-2-1124-13b-instruct",
+  "ornith-1-5-35b-a3b",
+  "ornith-1-5-9b",
+  "phi-3-5-mini-instruct",
+  "phi-4",
+  "phi-4-mini-instruct",
+  "qwen-agentworld-35b-a3b",
+  "qwen2-5-0-5b-instruct",
+  "qwen2-5-1-5b-instruct",
+  "qwen2-5-14b-instruct",
+  "qwen2-5-32b-instruct",
+  "qwen2-5-3b-instruct",
+  "qwen2-5-72b-instruct",
+  "qwen2-5-7b-instruct",
+  "qwen2-5-coder-32b-instruct",
+  "qwen2-5-coder-7b-instruct",
+  "qwen3-0-6b",
+  "qwen3-1-7b",
+  "qwen3-14b",
+  "qwen3-30b-a3b",
+  "qwen3-32b",
+  "qwen3-4b",
+  "qwen3-8b",
+  "qwq-32b",
+  "qwythos-9b-claude-mythos-5-1m",
+  "smollm2-1-7b-instruct",
+  "smollm2-135m-instruct",
+  "smollm2-360m-instruct",
+  "spark-x2-5-4b",
+  "tinyllama-1-1b-chat-v1-0",
+  "yi-1-5-34b-chat"
+]
  2. First observed

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does a good job: it enumerates the per-quantization output fields (weight bytes and basis, headroom, largest fitting context, decode ceiling), names the five-state verdict and best-quant selection, and discloses data provenance (each repo's config.json, tensor-shape index, the site's device table). It does not explicitly state that the call is read-only/non-mutating or whether results are deterministic or cached, so it falls short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first line, followed by output and provenance detail in a logical order, with no filler sentences. It is somewhat longer than strictly necessary and repeats the return detail that the output schema already supplies, keeping it out of the top band.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be re-explained, yet the description supplies useful sourcing and provenance context (config.json, device table, canonicalUrl) that the schema does not cover. Combined with the fully documented input schema, an agent has everything needed to call it, though read-only semantics remain implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters (including the batch/context/kvBytes defaults) are already fully documented in the schema, setting the baseline at 3. The description adds only adjacent context (the quantization range and provenance behavior) rather than new per-parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The title/description state a specific decision task ('decide whether one LLM fits one accelerator') plus an explicit scope ('at every quantization'). It also identifies the underlying instrument ('The identical function behind the /lab/llm-vram instrument'), making the tool's identity unambiguous. Siblings are unrelated domains, so no differentiation burden remains.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the use case clear (evaluate a model/GPU pair across quantization levels) and notes the published-instrument parity, but states no when-to-use/when-not conditions or any alternative approach. With no sibling overlap there is little to exclude, but the guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All MakerPortal compute toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources