A/B test peeking and sequential testing risk
check_peeking_riskWhat peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | Significance level, default 0.05. | |
| looks | Yes | How many times the result is checked, including the final look. Equally spaced. |