splithub_experiment_create
Add a new A/B test experiment to splithub.config.ts with variants, goals, traffic allocation, and status. Set it to running to go live on deployment or draft to hold it.
Instructions
Add an experiment to splithub.config.ts (checked by reloading the file). It is written as running: it goes live when the code is deployed (and pushed in cloud mode). Tell the user that, and that splithub_experiment_status paused holds it. Pass status draft only when the user asks to hold it. kind: value (text/number/URL read with useValue), component (/ markup), redirect (match path + per-variant destination), mvt (factors instead of variants, every combination compared, read with useValue as { factor: value }). First variant / first level is the control. In an app (React Native, Expo) redirect does nothing, and the config ships inside the build: the variants' code needs a release, then starting, pausing and weights apply live from the cloud; target new builds with an app_version gte condition in the flow. goodSample: total visitors at which the sample counts as good (default 5000). minDays: opt-in fewest days before a verdict (default 0 = none). Then wire the returned snippet, call splithub_doctor and splithub_refs. The loop: splithub_plan → splithub_experiment_create → wire the code → splithub_doctor → splithub_push (cloud) → splithub_results (re-check after daysToVerdict days) → splithub_experiment_status finished → splithub_ship.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | experiment key, lowercase | |
| to | No | redirect experiments: variant → destination path | |
| end | No | ISO date the experiment ends | |
| goal | No | the goal: any event name the app sends with track() (signup, Video Played) or a goal key from the config; an event not in goals becomes a goal, nothing to declare first | |
| kind | Yes | ||
| name | No | human name shown instead of the key; can be renamed later, the key stays | |
| layer | No | mutual exclusion: experiments in one layer never share a visitor; bucket ranges [start, end) must not overlap | |
| match | No | redirect experiments: the path it runs on, e.g. / | |
| start | No | ISO date the experiment starts | |
| status | No | ||
| values | No | value experiments: variant → value | |
| factors | No | mvt only: factor → level → value, e.g. { headline: { old: "Old", new: "New" }, button: { blue: "#2563eb", green: "#16a34a" } }; first level = control; at most 4 factors × 4 levels | |
| minDays | No | opt-in fewest days before a verdict (default 0 = none); verdicts otherwise depend on sample size | |
| traffic | No | percent of visitors in the experiment (default 100) | |
| weights | No | relative weights, same order as variants (not for mvt) | |
| variants | No | variant keys, first = control (not for mvt) | |
| secondary | No | goals (event names or goal keys) to watch next to the primary goal | |
| goodSample | No | total visitors at which verdict.sample.quality turns good (default 5000); informational, never changes the verdict | |
| guardrails | No | goals (event names or goal keys) that must not get worse; { goal, worse: "higher" } when a rise is bad (refunds, errors). A breach makes the verdict guardrail_breached: do not ship | |
| description | No | ||
| maxVisitors | No | stop counting after this many visitors |