batch_run
Run multiple prompts in parallel with bounded concurrency and auto-retry. Picks the fastest model, automatically retries failed items on alternatives, and supports incremental JSONL output resumable on interruption.
Instructions
Run multiple prompts in parallel with bounded concurrency and auto-retry.
For batch workloads: translation pipelines, data extraction, classification, content generation. Picks the fastest model and runs all prompts through it. Failed items are automatically retried on alternate models.
Each prompt dict should have a "prompt" key and optional "system_prompt" (overrides the top-level system_prompt) and "metadata" keys for tracking.
When results_file is set, each completed item is appended as a JSON line immediately. If interrupted, the file contains all completed items and can be resumed (already-completed indices are skipped).
Args: prompts: List of {"prompt": "...", "system_prompt": "...", "metadata": {...}} system_prompt: Default system prompt for all items (per-item overrides) model_id: Specific model to use (skips scanning). Use list_models() to browse. provider: Limit to a specific provider min_tier: Minimum quality tier when auto-selecting (default "A") free_only: If true, only use free models max_tokens: Max response tokens per item (default 4096) temperature: Sampling temperature (default 0.0) concurrency: Max parallel requests (default 5) retry_on_fail: Auto-retry failed items on alternate models (default true) results_file: Path to JSONL file for incremental writes and resume support
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| prompts | Yes | ||
| min_tier | No | A | |
| model_id | No | ||
| provider | No | ||
| free_only | No | ||
| max_tokens | No | ||
| concurrency | No | ||
| temperature | No | ||
| results_file | No | ||
| retry_on_fail | No | ||
| system_prompt | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |