ab_test_significance
Run a two-proportion z-test on two variants of an ad, landing page or audience, and report the p-value, whether the difference is statistically significant at the chosen confidence level, and how much more traffic would be needed to resolve it if it is not. Use it for any comparison of two rates instead of judging the gap by eye.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"variant_a_trials": "<variant_a_trials>",
"variant_a_successes": "<variant_a_successes>",
"variant_b_trials": "<variant_b_trials>",
"variant_b_successes": "<variant_b_successes>"
},
"name": "ab_test_significance"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| confidence | No | Confidence level for the test. | |
| variant_a_trials | Yes | Denominator for variant A: impressions for a click rate, clicks for a conversion rate. | |
| variant_b_trials | Yes | Denominator for variant B. | |
| variant_a_successes | Yes | Numerator for variant A. | |
| variant_b_successes | Yes | Numerator for variant B. |