Experiment Create
experiment_createModel Lab: run one prompt across many AI models at once and compare their outputs, cost, and latency side by side. Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (top up via experiment_topup); bills actual provider cost × 1.5 with a $0.10 minimum per experiment. ASYNC: returns experiment_id immediately — execution starts within ~1 minute; poll experiment_status until complete, then call experiment_results. Do NOT wait synchronously. Set summary:false to skip the AI-written model-comparison summary. Example: experiment_create({ name: "tone test", prompts: ["Rewrite formally: ..."], models: ["anthropic/claude-haiku-4.5", "openai/gpt-5-mini"], reps: 2, max_spend_usd: 2 })
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Short experiment name | |
| reps | No | Repetitions per prompt×model for variance, 1-5 (default 1) | |
| models | Yes | Model ids from experiment_models (max 12) | |
| params | No | Optional {system, temperature, max_tokens (default 512)} | |
| prompts | Yes | Prompts to test (max 20) | |
| summary | No | AI-written comparison of the models' outputs when the run completes (default true) | |
| max_spend_usd | Yes | REQUIRED hard spend cap in USD for this experiment (max 100). Execution stops when reached. |