HEAVY: Runs REAL inference. Measures a model on this machine with a warmup (absorbs load time) plus N streamed runs: time-to-first-token, generation tokens/sec (runtime-reported when available, else measured after the first token), and prompt-processing speed. Pass models[] to compare several models with the same prompt, ranked by generation speed.

DELEGATED INFERENCE: Offload a completion to a model you choose for cost savings and privacy. Without provider, runs on the live LOCAL provider that has the model (never a hosted API unless LOCAL_AI_ALLOW_HOSTED_FALLBACK is set); the result's 'provider', 'host' and 'local' fields say where it ran. Supports system prompts, JSON / JSON-Schema output (validated 'parsed' field), seed/topP/topK, image inputs for vision models, and contextLength for long inputs. Streams tokens via MCP progress notifications when the client sends a progressToken.

ONE CALL to offload work locally: pick the best model for a task on THIS machine, make it ready, and run it. Prefers models already loaded (no load cost), then installed models that fit your free VRAM, then (with allowPull) the best catalog model that fits. Task specialists (coder, vision, reasoning) win over general models; then larger fitting models win, grouped into size tiers (within ~20% parameter count); inside a tier, the model benchmarked fastest on this machine wins (run benchmark to record speeds; results expire after 14 days). Never uses a hosted provider unless allowHosted is true (or, when allowHosted is omitted, LOCAL_AI_ALLOW_HOSTED_FALLBACK is set), and even then any suitable local model is preferred. Models that only fit in system RAM run on CPU when nothing fits the GPU. Returns the completion plus a 'routing' block explaining the choice.

One read-only snapshot for troubleshooting: the effective configuration (API keys never shown, only whether one is set), every configured provider with its detection result, latency and error, the benchmark speed cache, and how hardware was probed. Paste its output into bug reports.

DELEGATED EMBEDDINGS: Generate embeddings for a string or an array of texts. Without provider, runs on the live local provider that has the model. Always returns one vector per input.

Make a model ready in one call: optionally pull it, check it fits current free VRAM, unload other models if needed (evict: 'lru' or 'all'), load it, and verify. Use dryRun:true to get the plan without changing anything.

Will a model run on this machine, and where? Uses the installed model's size and exact layer architecture when the runtime exposes it (else parameter-count or catalog estimates) to size weights + KV cache for a context length, then compares with free VRAM summed across GPUs and available RAM. Returns target gpu / partial (GPU+CPU split) / cpu / none, with the numbers and how the KV estimate was made. Credits back memory the model already uses if it is loaded.

Check whether each provider is reachable and report its version (and live metrics such as vLLM KV-cache usage). Without provider, checks every configured provider, including hosted ones.

List models currently loaded into memory on each live provider, with VRAM use and context length where the runtime reports them.

List models installed/available on each live provider, keyed by provider. Pass provider to query just one.

List configured model runtimes (Ollama, LM Studio, llama.cpp, Docker Model Runner, and any opt-in MLX / vLLM / Foundry Local / OpenAI-compatible / hosted instances) with host, kind (local or hosted), live status, and capabilities.

Load a model into memory on ONE provider so it is ready for inference. Optional keepAlive (e.g. '5m', '1h') and contextLength. Without provider, uses the live local provider that has the model.

Detailed metadata for one model (family, parameter size, quantization, context length, and layer architecture where available). Without provider, queries the live local providers that have the model.

Stop a background pull started with pull_model { background: true }. Finished jobs are returned unchanged.

HEAVY: Download a model onto ONE provider (multiple GB, can take a long time). Streams download progress via MCP progress notifications when the client sends a progressToken. With background:true, returns a jobId immediately instead; poll pull_status and stop it with pull_cancel (use this when the client has no progress display or a short tool timeout). Without provider, uses the provider that already has the model, else the first live local provider that can pull.

Status of background pulls started with pull_model { background: true }: state (running, done, failed, cancelled), progress, and result or error. Without jobId, lists every job this server remembers (finished jobs are kept for an hour; jobs are forgotten on restart).

DESTRUCTIVE: Permanently delete a model from one provider. provider is REQUIRED (no fan-out). If the client supports elicitation the user is asked to confirm directly; otherwise confirm:true is required.

Search the built-in curated catalog of well-known local models (Ollama tags) by name, family, or task. Offline and instant. For live results with per-quant fit, use search_models with source 'huggingface'.

Find models to download, annotated with whether each fits this machine. source 'huggingface' searches GGUF repos on the Hugging Face Hub and returns per-quant sizes, the best quant for this hardware, and pullable references (Ollama: hf.co/:). source 'catalog' searches the offline curated list. The Ollama library has no public search API, so it is not a source.

Recommend catalog models for a task, ranked for THIS machine: fully-on-GPU first, then partial GPU offload, then CPU-only. Within a tier the largest (most capable) model comes first unless preferSpeed is true. Flags models already installed on a live provider and explains each pick.

Report local hardware: platform, total/available RAM, CPU count, and GPUs with total/free VRAM (NVIDIA via nvidia-smi, AMD/Intel via sysfs or the Windows registry, Apple Silicon unified memory). Each GPU names its data source.

Unload a model from memory to free VRAM/RAM on ONE provider. Without provider, uses the live local provider that has it loaded.