Delegate a task to the best local model
delegateOffload a task to a suitable local AI model in one call: selects, prepares, and runs a model for the task, then returns the completion and routing details.
Instructions
ONE CALL to offload work locally: pick the best model for a task on THIS machine, make it ready, and run it. Prefers models already loaded (no load cost), then installed models that fit your free VRAM, then (with allowPull) the best catalog model that fits. Task specialists (coder, vision, reasoning) win over general models; then larger fitting models win, grouped into size tiers (within ~20% parameter count); inside a tier, the model benchmarked fastest on this machine wins (run benchmark to record speeds; results expire after 14 days). Never uses a hosted provider unless allowHosted is true (or, when allowHosted is omitted, LOCAL_AI_ALLOW_HOSTED_FALLBACK is set), and even then any suitable local model is preferred. Models that only fit in system RAM run on CPU when nothing fits the GPU. Returns the completion plus a 'routing' block explaining the choice.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"task": "<task>"
},
"name": "delegate"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| seed | No | Seed for reproducible sampling | |
| stop | No | Stop sequences | |
| task | Yes | What kind of work this is | |
| topK | No | ||
| topP | No | ||
| prompt | No | Plain prompt text (alternative to messages) | |
| stream | No | Stream tokens (default true). Progress notifications are sent when the client provides a progressToken. | |
| system | No | System instruction | |
| options | No | Provider-specific extras passed through as-is | |
| fallback | No | If the chosen model fails before producing output (load error, runtime crash, timeout), try the next ranked candidate, up to 3 attempts. Failed attempts are listed in routing.attempts (default false) | |
| messages | No | Chat messages; content may be a string or text/image parts (vision models) | |
| provider | No | Restrict to one provider. Omit to let the server choose (local providers only, unless LOCAL_AI_ALLOW_HOSTED_FALLBACK is set). | |
| allowPull | No | Download a catalog model if nothing suitable is installed (default false) | |
| maxTokens | No | Maximum tokens to generate | |
| allowHosted | No | Allow hosted providers as a fallback after local ones. false always excludes them; omitted follows LOCAL_AI_ALLOW_HOSTED_FALLBACK | |
| temperature | No | Sampling temperature | |
| maxSizeBytes | No | Ignore models larger than this | |
| contextLength | No | Context window to run with (Ollama num_ctx). Raise it for long inputs to avoid truncation. | |
| repeatPenalty | No | ||
| repairAttempts | No | With a json_schema responseFormat: if the output fails validation, send the errors back to the same model and ask for corrected JSON, up to this many times (default 0, max 2). Token usage covers every attempt. | |
| responseFormat | No | Constrain output to JSON (json_object) or a JSON Schema (json_schema); the result includes a validated 'parsed' value |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| model | Yes | ||
| routing | Yes | ||
| provider | Yes |