Check sample ratio
check_srmTests observed arm counts against intended weights with a chi-square check to detect sample ratio mismatch in experiments.
Instructions
Chi-square test of observed arm counts against intended weights.
A mismatch means the counts are unlikely given the weights. It does not identify the cause.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | Mismatch threshold. 0.001 is the usual SRM bar. | |
| counts | Yes | Observed assignments per arm. | |
| expected_weights | Yes | Intended traffic weights. Must sum to 1 and match the count keys. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| alpha | Yes | ||
| counts | Yes | ||
| p_value | Yes | ||
| mismatch | Yes | ||
| chi_square | Yes | ||
| expected_counts | Yes | ||
| expected_weights | Yes | ||
| degrees_of_freedom | Yes |