Dispatch many prompts to local models
ollama_dispatch_batchRun many generations in one call, grouping by model to avoid reload overhead, returning results in input order with isolated failures.
Instructions
Run many generations in one call. Items are grouped by resolved model and the groups run sequentially, so a cold model load is paid at most once per model instead of thrashing VRAM. Results are returned in input order regardless of execution order, and one failing item never voids the run. Prefer this over many separate ollama_dispatch calls: it is faster and costs you far less context.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| host | No | Override the base URL of the local inference server for this call only, e.g. "http://localhost:11434". Omit to use the configured host. | |
| items | Yes | The work list, 1-64 items. Results always come back in THIS order regardless of execution order, and a failed item is reported as a failure without voiding the rest. | |
| model | No | Model selector. Three grammars are accepted: (1) a literal installed model name or configured alias, used as-is; (2) "role:NAME" — resolve through the named role, which carries an ordered fallback chain plus its own sampling defaults; (3) "caps:a+b" — pick the best installed model advertising ALL of the named capabilities (e.g. "caps:vision+tools"), ranked by the configured policy. Omit this field entirely to use the configured default role, which is the right choice unless you have a reason. Selectors never name a model in server code — availability is discovered at runtime, so an unknown or un-pulled name is an error, not a silent substitution. | |
| detail | No | Response verbosity. "concise" returns the answer plus minimal provenance; "detailed" adds the resolution trail, token counts, timings and warnings. Default is the configured value. | |
| defaults | No | Per-item settings applied wherever the item itself is silent. | |
| keep_alive | No | How long the model stays resident in VRAM after this call: a duration string such as "10m", or seconds as a number. 0 unloads immediately; a negative number keeps it loaded indefinitely. Keeping a model warm avoids re-paying a multi-second load on the next call. | |
| timeout_ms | No | Per-request timeout in milliseconds. Large models on a cold load can take tens of seconds before the first token, so prefer generous values over retrying. | |
| concurrency | No | How many items of the SAME model run at once. Default 1, which is the VRAM-safe choice; raise it only for small models on a machine with headroom. Groups themselves always run one after another. | |
| stop_on_error | No | Abort remaining items after the first failure. Off by default — partial results beat no results, and unstarted items are reported as such. | |
| max_output_chars | No | Hard cap on the number of characters returned to you. Output beyond this is trimmed and the trim is reported, never hidden. Use it to protect your own context window. |