compare
Compare a shipped change and return a keep, revert, or extend verdict. Compare is a free read. Each successful call counts as one metered read. Pass ship_date plus window_days, or explicit baseline and compare windows. window_days is an integer from 1 to 90 and a multiple of 7 (default 7, so the largest ship window is 84). The ship day is in neither window: baseline is ship-N through ship-1, and compare is ship+1 through ship+N. Run a day-7 read on ship day + 8 so the last day is complete. If either window includes today UTC, the verdict is extend with reason incomplete_current_day, and that reason overrides every other verdict. goal_event is required and is counted as unique converters (count_mode unique_users): a user who fires the goal more than once counts once. denominator_event defaults to page_view and is counted as unique users. rate = goal_converters / denominator_unique_users. Optional filters use the query filter shape and apply as they do today. A filter on event is rejected. Optional denominator_filters use the same {field, op, value} shape and operators and apply only to denominator events when computing denominator_unique_users. goal_converters still counts only goal users inside that filtered denominator. Omit denominator_filters or pass [] for today's behavior. Max 10 entries. To exclude one-hit 404 page views from the denominator: denominator_filters [{"field":"properties.title","op":"neq","value":"Page not found | grokbotpages.com"}]. rule thresholds are relative fractions (0.15 means +15%). The absolute value of keep_threshold and revert_threshold must be at most 5. Omitted rule fields use the documented day-7 examples (keep_threshold 0.15, revert_threshold -0.1, min_denominator 200, min_goal_events 20 unique converters across both windows). Those examples are not product policy. Verdict order: extend incomplete_current_day; extend below_min_sample; extend zero_denominator; extend zero_baseline when lift is null; revert revert_threshold_met only when lift is at or below revert_threshold and the 90% interval lies entirely below zero; extend inconclusive when the drop is within noise; keep keep_threshold_met when lift is at or above keep_threshold and interval.low is above 0; extend interval_includes_zero; extend no_evidence_of_gain. A drop by itself is not a revert. Each window reports date_from, date_to, day_count, denominator_unique_users, goal_unique_users, goal_converters, goal_users_outside_denominator, count_mode, and rate. The response also includes count_mode, reason_detail, lift, echoed denominator_filters, and a 90% interval (level, method newcombe_wilson_90, estimate rate_difference, low, high, excludes_zero, note). Optional metric compares a numeric property. metric is sum(), avg(), p50(), p75(), or p95(), for example sum(revenue) or p75(cwv_lcp). The property uses the query property allowlist ^[A-Za-z0-9_]+$ with a maximum of 128 characters. Example: metric p75(cwv_lcp), metric_event page_view, better lower, while goal_event stays signup. On hosted, cwv_* vitals are properties on page_view. Prefer the web_vitals tool for the p75 headline, ratings, and shares. metric_event is the event whose property is measured and defaults to goal_event. Set it when the measured event is not the goal. better is higher (default) or lower and is not inferred from the property name. Use better=lower for cwv_lcp and the other Web Vitals. Shared filters apply to the metric event. denominator_filters still apply only to the denominator. sum() compares the sum divided by unique denominator users (per_user true). avg and percentiles compare the aggregate itself (per_user false). Percentiles are nearest-rank, so percentile_method is nearest_rank. With metric set, interval is null and interval_method is none. There is no bootstrap interval in v1. lift is (compare - baseline) / baseline, negated when better is lower, so the existing thresholds keep their meaning. raw_lift is the unnegated ratio. min_denominator is still unique denominator users per window. When metric is set, min_goal_events is numeric values (n) on metric_event across both windows, not unique converters. below_min_sample is extend when n across both windows is under min_goal_events (default 20) or either window is under min_denominator (default 200). incomplete_current_day, below_min_sample, zero_denominator, and zero_baseline stay in that order. A missing compare value, or a baseline of 0 or null, is zero_baseline with lift null. When lift meets keep_threshold or revert_threshold, the verdict is extend with reason no_interval_for_metric, and lift is still reported. Lift between the thresholds is extend / no_evidence_of_gain. The response metric object includes metric, fn, property, metric_event, better, per_user, and percentile_method. Each window still includes the conversion rate and user counts, plus value, n, and skipped. Omit metric and the compare response is unchanged, including the Newcombe interval. Compare stays a free read when metric is set.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"project": "<project>",
"goal_event": "<goal_event>"
},
"name": "compare"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| rule | No | Decision rule. Omitted fields use the documented day-7 examples, not product policy. Threshold absolute values above 5 are rejected | |
| better | No | Which direction is an improvement. Defaults to higher and is not inferred from the property name. Use better=lower for cwv_lcp and the other Web Vitals | |
| metric | No | Numeric property metric, for example p75(cwv_lcp) or sum(revenue). One of sum, avg, p50, p75, p95. Omit to keep the conversion-rate compare | |
| compare | No | Explicit compare window. Pass with baseline, and omit ship_date and window_days | |
| filters | No | Optional filters, same shape as query. Do not filter on event | |
| project | Yes | Project name or ID | |
| baseline | No | Explicit baseline window. Pass with compare, and omit ship_date and window_days | |
| ship_date | No | Ship date (YYYY-MM-DD, UTC). The ship day is in neither window | |
| goal_event | Yes | Goal event counted as unique converters (count_mode unique_users). A user who fires it more than once counts once | |
| window_days | No | Days in each ship_date window. Integer from 1 to 90 and a multiple of 7. Defaults to 7. Omit when passing baseline and compare | |
| metric_event | No | Event whose property is measured. Defaults to goal_event. For hosted Web Vitals such as p75(cwv_lcp), set this to page_view | |
| denominator_event | No | Denominator event counted as unique users. Defaults to page_view. rate = goal_converters / denominator_unique_users | |
| denominator_filters | No | Filters applied only to denominator events when counting denominator_unique_users. Same shape and operators as filters. Max 10. Omit or pass [] for today's behavior. Example: {"field":"properties.title","op":"neq","value":"Page not found | grokbotpages.com"} |