Skip to main content
Glama

compare

Read-only

Compare a shipped change and return a keep, revert, or extend verdict. Compare is a free read. Each successful call counts as one metered read. Pass ship_date plus window_days, or explicit baseline and compare windows. window_days is an integer from 1 to 90 and a multiple of 7 (default 7, so the largest ship window is 84). The ship day is in neither window: baseline is ship-N through ship-1, and compare is ship+1 through ship+N. Run a day-7 read on ship day + 8 so the last day is complete. If either window includes today UTC, the verdict is extend with reason incomplete_current_day, and that reason overrides every other verdict. goal_event is required and is counted as unique converters (count_mode unique_users): a user who fires the goal more than once counts once. denominator_event defaults to page_view and is counted as unique users. rate = goal_converters / denominator_unique_users. Optional filters use the query filter shape and apply as they do today. A filter on event is rejected. Optional denominator_filters use the same {field, op, value} shape and operators and apply only to denominator events when computing denominator_unique_users. goal_converters still counts only goal users inside that filtered denominator. Omit denominator_filters or pass [] for today's behavior. Max 10 entries. To exclude one-hit 404 page views from the denominator: denominator_filters [{"field":"properties.title","op":"neq","value":"Page not found | grokbotpages.com"}]. rule thresholds are relative fractions (0.15 means +15%). The absolute value of keep_threshold and revert_threshold must be at most 5. Omitted rule fields use the documented day-7 examples (keep_threshold 0.15, revert_threshold -0.1, min_denominator 200, min_goal_events 20 unique converters across both windows). Those examples are not product policy. Verdict order: extend incomplete_current_day; extend below_min_sample; extend zero_denominator; extend zero_baseline when lift is null; revert revert_threshold_met only when lift is at or below revert_threshold and the 90% interval lies entirely below zero; extend inconclusive when the drop is within noise; keep keep_threshold_met when lift is at or above keep_threshold and interval.low is above 0; extend interval_includes_zero; extend no_evidence_of_gain. A drop by itself is not a revert. Each window reports date_from, date_to, day_count, denominator_unique_users, goal_unique_users, goal_converters, goal_users_outside_denominator, count_mode, and rate. The response also includes count_mode, reason_detail, lift, echoed denominator_filters, and a 90% interval (level, method newcombe_wilson_90, estimate rate_difference, low, high, excludes_zero, note). Optional metric compares a numeric property. metric is sum(), avg(), p50(), p75(), or p95(), for example sum(revenue) or p75(cwv_lcp). The property uses the query property allowlist ^[A-Za-z0-9_]+$ with a maximum of 128 characters. Example: metric p75(cwv_lcp), metric_event page_view, better lower, while goal_event stays signup. On hosted, cwv_* vitals are properties on page_view. Prefer the web_vitals tool for the p75 headline, ratings, and shares. metric_event is the event whose property is measured and defaults to goal_event. Set it when the measured event is not the goal. better is higher (default) or lower and is not inferred from the property name. Use better=lower for cwv_lcp and the other Web Vitals. Shared filters apply to the metric event. denominator_filters still apply only to the denominator. sum() compares the sum divided by unique denominator users (per_user true). avg and percentiles compare the aggregate itself (per_user false). Percentiles are nearest-rank, so percentile_method is nearest_rank. With metric set, interval is null and interval_method is none. There is no bootstrap interval in v1. lift is (compare - baseline) / baseline, negated when better is lower, so the existing thresholds keep their meaning. raw_lift is the unnegated ratio. min_denominator is still unique denominator users per window. When metric is set, min_goal_events is numeric values (n) on metric_event across both windows, not unique converters. below_min_sample is extend when n across both windows is under min_goal_events (default 20) or either window is under min_denominator (default 200). incomplete_current_day, below_min_sample, zero_denominator, and zero_baseline stay in that order. A missing compare value, or a baseline of 0 or null, is zero_baseline with lift null. When lift meets keep_threshold or revert_threshold, the verdict is extend with reason no_interval_for_metric, and lift is still reported. Lift between the thresholds is extend / no_evidence_of_gain. The response metric object includes metric, fn, property, metric_event, better, per_user, and percentile_method. Each window still includes the conversion rate and user counts, plus value, n, and skipped. Omit metric and the compare response is unchanged, including the Newcombe interval. Compare stays a free read when metric is set.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "project": "<project>",
      "goal_event": "<goal_event>"
    },
    "name": "compare"
  }
}

Input Schema

NameRequiredDescriptionDefault
ruleNoDecision rule. Omitted fields use the documented day-7 examples, not product policy. Threshold absolute values above 5 are rejected
betterNoWhich direction is an improvement. Defaults to higher and is not inferred from the property name. Use better=lower for cwv_lcp and the other Web Vitals
metricNoNumeric property metric, for example p75(cwv_lcp) or sum(revenue). One of sum, avg, p50, p75, p95. Omit to keep the conversion-rate compare
compareNoExplicit compare window. Pass with baseline, and omit ship_date and window_days
filtersNoOptional filters, same shape as query. Do not filter on event
projectYesProject name or ID
baselineNoExplicit baseline window. Pass with compare, and omit ship_date and window_days
ship_dateNoShip date (YYYY-MM-DD, UTC). The ship day is in neither window
goal_eventYesGoal event counted as unique converters (count_mode unique_users). A user who fires it more than once counts once
window_daysNoDays in each ship_date window. Integer from 1 to 90 and a multiple of 7. Defaults to 7. Omit when passing baseline and compare
metric_eventNoEvent whose property is measured. Defaults to goal_event. For hosted Web Vitals such as p75(cwv_lcp), set this to page_view
denominator_eventNoDenominator event counted as unique users. Defaults to page_view. rate = goal_converters / denominator_unique_users
denominator_filtersNoFilters applied only to denominator events when counting denominator_unique_users. Same shape and operators as filters. Max 10. Omit or pass [] for today's behavior. Example: {"field":"properties.title","op":"neq","value":"Page not found | grokbotpages.com"}

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / metric_event / description
      Previous value: -"Event whose property is measured. Defaults to goal_event. For p75(cwv_lcp), set the event properties_received shows: page_view on some sites, or $web_vitals when data-track-vitals is on"New value: +"Event whose property is measured. Defaults to goal_event. For hosted Web Vitals such as p75(cwv_lcp), set this to page_view"
  2. Changed4 schema fields changed
    • addedInput schema / properties / better
      Added value: +{
      +  "description": "Which direction is an improvement. Defaults to higher and is not inferred from the property name. Use better=lower for cwv_lcp and the other Web Vitals",
      +  "enum": [
      +    "higher",
      +    "lower"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / metric
      Added value: +{
      +  "description": "Numeric property metric, for example p75(cwv_lcp) or sum(revenue). One of sum, avg, p50, p75, p95. Omit to keep the conversion-rate compare",
      +  "pattern": "^(sum|avg|p50|p75|p95)\\([A-Za-z0-9_]{1,128}\\)$",
      +  "type": "string"
      +}
    • addedInput schema / properties / metric_event
      Added value: +{
      +  "description": "Event whose property is measured. Defaults to goal_event. For p75(cwv_lcp), set the event properties_received shows: page_view on some sites, or $web_vitals when data-track-vitals is on",
      +  "maxLength": 256,
      +  "minLength": 1,
      +  "type": "string"
      +}
    • changedInput schema / properties / rule / properties / min_goal_events / description
      Previous value: -"Minimum unique goal converters across both windows. The field name stays min_goal_events"New value: +"Minimum unique goal converters across both windows when metric is omitted. When metric is set, numeric values (n) on metric_event across both windows. The field name stays min_goal_events"
  3. Changed1 schema field changed
    • addedInput schema / properties / denominator_filters
      Added value: +{
      +  "description": "Filters applied only to denominator events when counting denominator_unique_users. Same shape and operators as filters. Max 10. Omit or pass [] for today's behavior. Example: {\"field\":\"properties.title\",\"op\":\"neq\",\"value\":\"Page not found | grokbotpages.com\"}",
      +  "items": {
      +    "properties": {
      +      "field": {
      +        "description": "Field to filter on (user_id, date, country, session_id, timestamp, or properties.*). event is rejected",
      +        "type": "string"
      +      },
      +      "op": {
      +        "description": "Filter operator",
      +        "enum": [
      +          "eq",
      +          "neq",
      +          "gt",
      +          "lt",
      +          "gte",
      +          "lte",
      +          "contains"
      +        ],
      +        "type": "string"
      +      },
      +      "value": {
      +        "description": "Value to compare against",
      +        "type": [
      +          "string",
      +          "number"
      +        ]
      +      }
      +    },
      +    "required": [
      +      "field",
      +      "op",
      +      "value"
      +    ],
      +    "type": "object"
      +  },
      +  "maxItems": 10,
      +  "type": "array"
      +}
  4. Added

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly and non-destructive annotations, the description explains that this is a free read that still counts as a metered read. It also discloses verdict precedence, incomplete-day behavior, sample thresholds, zero-denominator handling, interval behavior, and metric-specific differences at length.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded and nearly all content is relevant to this complex tool, but the description is a very long unbroken block with repetitive points such as repeated statements that compare remains a free read. It would be easier to scan if structured into sections for windows, verdict order, metrics, and output.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 13-parameter tool with nested objects and no output schema, the description is unusually complete. It covers defaults, edge cases, metric semantics, verdict ordering, response fields, and interval behavior, so an agent has enough information to call and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds substantial relational meaning the schema cannot convey: the ship day is excluded from both windows, baseline and compare boundaries are ship-N through ship-1 and ship+1 through ship+N, denominator_filters apply only to denominator events, and metric changes min_goal_events from unique converters to numeric values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Compare a shipped change and return a keep, revert, or extend verdict.' It also distinguishes the tool from web_vitals for p75 metric use, so an agent can identify the tool's role without inspecting the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit invocation paths: ship_date plus window_days, or explicit baseline and compare windows. It also states when to prefer web_vitals and gives a concrete timing rule: run a day-7 read on ship day + 8 so the last day is complete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources