Skip to main content
Glama
Local AI MCP

Local AI MCP

Official
by TMHSDigital

Delegate a task to the best local model

delegate

Offload a task to a suitable local AI model in one call: selects, prepares, and runs a model for the task, then returns the completion and routing details.

Instructions

ONE CALL to offload work locally: pick the best model for a task on THIS machine, make it ready, and run it. Prefers models already loaded (no load cost), then installed models that fit your free VRAM, then (with allowPull) the best catalog model that fits. Task specialists (coder, vision, reasoning) win over general models; then larger fitting models win, grouped into size tiers (within ~20% parameter count); inside a tier, the model benchmarked fastest on this machine wins (run benchmark to record speeds; results expire after 14 days). Never uses a hosted provider unless allowHosted is true (or, when allowHosted is omitted, LOCAL_AI_ALLOW_HOSTED_FALLBACK is set), and even then any suitable local model is preferred. Models that only fit in system RAM run on CPU when nothing fits the GPU. Returns the completion plus a 'routing' block explaining the choice.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "task": "<task>"
    },
    "name": "delegate"
  }
}

Input Schema

NameRequiredDescriptionDefault
seedNoSeed for reproducible sampling
stopNoStop sequences
taskYesWhat kind of work this is
topKNo
topPNo
promptNoPlain prompt text (alternative to messages)
streamNoStream tokens (default true). Progress notifications are sent when the client provides a progressToken.
systemNoSystem instruction
optionsNoProvider-specific extras passed through as-is
fallbackNoIf the chosen model fails before producing output (load error, runtime crash, timeout), try the next ranked candidate, up to 3 attempts. Failed attempts are listed in routing.attempts (default false)
messagesNoChat messages; content may be a string or text/image parts (vision models)
providerNoRestrict to one provider. Omit to let the server choose (local providers only, unless LOCAL_AI_ALLOW_HOSTED_FALLBACK is set).
allowPullNoDownload a catalog model if nothing suitable is installed (default false)
maxTokensNoMaximum tokens to generate
allowHostedNoAllow hosted providers as a fallback after local ones. false always excludes them; omitted follows LOCAL_AI_ALLOW_HOSTED_FALLBACK
temperatureNoSampling temperature
maxSizeBytesNoIgnore models larger than this
contextLengthNoContext window to run with (Ollama num_ctx). Raise it for long inputs to avoid truncation.
repeatPenaltyNo
repairAttemptsNoWith a json_schema responseFormat: if the output fails validation, send the errors back to the same model and ask for corrected JSON, up to this many times (default 0, max 2). Token usage covers every attempt.
responseFormatNoConstrain output to JSON (json_object) or a JSON Schema (json_schema); the result includes a validated 'parsed' value

Output Schema

NameRequiredDescriptionDefault
textYes
modelYes
routingYes
providerYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changedv0.6.1
    • changedInput schema / properties / allowHosted / description
      Previous value: -"Allow hosted providers (default false)"New value: +"Allow hosted providers as a fallback after local ones. false always excludes them; omitted follows LOCAL_AI_ALLOW_HOSTED_FALLBACK"
    • addedInput schema / properties / fallback
      Added value: +{
      +  "description": "If the chosen model fails before producing output (load error, runtime crash, timeout), try the next ranked candidate, up to 3 attempts. Failed attempts are listed in routing.attempts (default false)",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / repairAttempts
      Added value: +{
      +  "description": "With a json_schema responseFormat: if the output fails validation, send the errors back to the same model and ask for corrected JSON, up to this many times (default 0, max 2). Token usage covers every attempt.",
      +  "maximum": 2,
      +  "minimum": 0,
      +  "type": "integer"
      +}
  2. Addedv0.5.0

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the safety profile (not read-only, open-world, not idempotent), and the description adds substantial behavior beyond that: the full routing preference order, the no-load-cost preference, VRAM-fit and CPU-fallback behavior, the 14-day benchmark expiry, and the env var controlling hosted fallback. It also discloses that the response carries a 'routing' block, which the annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the headline 'ONE CALL' and then organized by ranking rules, hosted policy and return shape; nearly every clause carries information. Minor redundancy where allowHosted's env-var behavior is restated in both the description and the schema, and the paragraph is dense enough to be slightly hard to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 21-parameter composite tool with an output schema, the description covers the decision logic an agent needs to call it correctly, and correctly defers return-shape details to the schema. It does not note that one of prompt or messages is effectively required, which is the main remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 86%, so the baseline is 3, but the description adds meaning that the schema lacks: how the task enum maps to specialist vs general model tiers, the ~20% size-tier grouping and benchmark tie-break, and the interaction between allowPull/allowHosted and local availability. Parameters such as maxSizeBytes, contextLength and repairAttempts are not elaborated beyond their schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: 'ONE CALL to offload work locally: pick the best model for a task on THIS machine, make it ready, and run it.' This distinguishes the composite routing tool from single-purpose siblings like complete, suggest_model, ensure_model and load_model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit selection conditions for hosted vs local ('Never uses a hosted provider unless allowHosted is true...') and the allowPull gate, plus points to the benchmark sibling for recording speeds. It does not explicitly say when to prefer a sibling such as complete or suggest_model instead, leaving that to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.