Skip to main content
Glama

mureo_outcome_evaluate

Evaluate whether a logged advertising action improved, regressed, or is inconclusive by comparing before/after metrics; it returns a reproducible verdict for observation-window reviews.

Instructions

Deterministically evaluate whether a logged action's outcome improved, regressed, or is inconclusive — the reproducible verdict the observation-window review (daily-check) and /learn rely on, instead of eyeballing the numbers. Pass before (typically the action_log entry's metrics_at_action) and after (the current numbers). Pure calculation — works for ANY platform (google_ads / meta_ads / tiktok_ads / plugins) as long as you feed comparable metric names. The verdict is ONE primary metric's: cpa when both sides have conversions, else 'inconclusive' (reason no_primary_metric); every other metric is a reference row. Synonyms merge first (cost_per_conversion is cpa, cost is spend). The primary is judged only when both sides reach the volume floor (conversions 30 for cpa/cpl/roas/conversions, impressions 1000 for ctr/cpm, clicks 200 for cpc/cvr; else insufficient_volume / volume_unknown). A zero baseline is 'inconclusive' (zero_baseline). Count metrics are compared per day using each side's days or period (YESTERDAY / LAST_7_DAYS / LAST_30_DAYS). A change within ±noise_pct (default 10%) is 'inconclusive' (no_change). The floors and the band come from STRATEGY.md ## Guardrails (outcome_noise_pct, outcome_min_conversions, outcome_min_impressions, outcome_min_clicks). To close an observation, pass evaluation_of (the entry's index): the entry's baseline, its decision's expected effects and its campaign's bidding target are read from STATE.json, exactly as the automatic closure reads them; before is refused beside a recorded baseline (evaluation_of_conflicts_with_before), and a result with no metric on both sides is refused (no_comparable_metric). The answer's evaluation is the table to append, unchanged, with evaluation_of via mureo_state_action_log_append.

Input Schema

NameRequiredDescriptionDefault
pathNoOptional path to the file. Defaults to STRATEGY.md / STATE.json in the MCP server's current working directory. Paths outside cwd are refused.
afterYesCurrent metrics, same shape as ``before``. With ``evaluation_of``, pass the campaign's ``metrics`` object from mureo_state_get unchanged, ``period`` and ``fetched_at`` included: ``fetched_at`` is the end of the table's window and ``period`` is what the count rule compares.
beforeNoBaseline metrics — metric name → number (e.g. {"cpa": 5000, "conversions": 50, "days": 7}). Usually the action_log entry's metrics_at_action. ``days`` / ``period`` say which window the figures cover; they are read, never scored. Required without ``evaluation_of``; with it, required when the entry recorded no metrics_at_action and refused when it recorded one (evaluation_of_conflicts_with_before).
expectedNoWithout evaluation_of only: the effects the change should have (e.g. [{"metric": "cpa", "direction": "down"}]); with several, mark exactly one "primary": true. The verdict is then read off that primary.
noise_pctNoNoise band in percent, above 0 and at most 100 (0, 100]. Omit it to use the workspace's ``outcome_noise_pct`` guardrail (default 10); pass it only to override that for this one call. A change smaller than this is 'inconclusive' (day-to-day variance).
evaluation_ofNoPositional index (into the full action_log) of the entry being evaluated — the pending entry's ``index``. Its metrics_at_action is the baseline, so ``before`` is refused beside it (evaluation_of_conflicts_with_before) unless the entry recorded none; its decision's expected effects apply, so ``expected`` is refused beside it (evaluation_of_conflicts_with_expected).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed7 schema fields changedv0.33.0
    • changedInput schema / properties / after / description
      Previous value: -"Current metrics, same shape as ``before``."New value: +"Current metrics, same shape as ``before``. With ``evaluation_of``, pass the campaign's ``metrics`` object from mureo_state_get unchanged, ``period`` and ``fetched_at`` included: ``fetched_at`` is the end of the table's window and ``period`` is what the count rule compares."
    • changedInput schema / properties / before / description
      Previous value: -"Baseline metrics — metric name → number (e.g. {\"cpa\": 5000, \"conversions\": 50}). Usually the action_log entry's metrics_at_action."New value: +"Baseline metrics — metric name → number (e.g. {\"cpa\": 5000, \"conversions\": 50, \"days\": 7}). Usually the action_log entry's metrics_at_action. ``days`` / ``period`` say which window the figures cover; they are read, never scored. Required without ``evaluation_of``; with it, required when the entry recorded no metrics_at_action and refused when it recorded one (evaluation_of_conflicts_with_before)."
    • addedInput schema / properties / evaluation_of
      Added value: +{
      +  "description": "Positional index (into the full action_log) of the entry being evaluated — the pending entry's ``index``. Its metrics_at_action is the baseline, so ``before`` is refused beside it (evaluation_of_conflicts_with_before) unless the entry recorded none; its decision's expected effects apply, so ``expected`` is refused beside it (evaluation_of_conflicts_with_expected).",
      +  "type": "integer"
      +}
    • addedInput schema / properties / expected
      Added value: +{
      +  "description": "Without evaluation_of only: the effects the change should have (e.g. [{\"metric\": \"cpa\", \"direction\": \"down\"}]); with several, mark exactly one \"primary\": true. The verdict is then read off that primary.",
      +  "items": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "direction": {
      +        "enum": [
      +          "up",
      +          "down",
      +          "hold"
      +        ],
      +        "type": "string"
      +      },
      +      "horizon_days": {
      +        "maximum": 365,
      +        "minimum": 1,
      +        "type": "integer"
      +      },
      +      "metric": {
      +        "enum": [
      +          "cpa",
      +          "cpc",
      +          "cpl",
      +          "cpm",
      +          "conversions",
      +          "cvr",
      +          "ctr",
      +          "roas",
      +          "cost",
      +          "spend",
      +          "clicks",
      +          "impressions"
      +        ],
      +        "type": "string"
      +      },
      +      "primary": {
      +        "description": "The effect the verdict is read off. One effect is primary by itself; with several, mark exactly one.",
      +        "type": "boolean"
      +      },
      +      "target": {
      +        "type": "number"
      +      }
      +    },
      +    "required": [
      +      "metric",
      +      "direction"
      +    ],
      +    "type": "object"
      +  },
      +  "maxItems": 12,
      +  "type": "array"
      +}
    • changedInput schema / properties / noise_pct / description
      Previous value: -"Noise band in percent (default 10). A change smaller than this is 'inconclusive' (day-to-day variance)."New value: +"Noise band in percent, above 0 and at most 100 (0, 100]. Omit it to use the workspace's ``outcome_noise_pct`` guardrail (default 10); pass it only to override that for this one call. A change smaller than this is 'inconclusive' (day-to-day variance)."
    • addedInput schema / properties / path
      Added value: +{
      +  "description": "Optional path to the file. Defaults to STRATEGY.md / STATE.json in the MCP server's current working directory. Paths outside cwd are refused.",
      +  "type": "string"
      +}
    • changedInput schema / required
      Previous value: -[
      -  "before",
      -  "after"
      -]New value: +[
      +  "after"
      +]
  2. Changed1 schema field changedv0.10.37
    • addedInput schema / additionalProperties
      Added value: +false
  3. Addedv0.10.17

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully carries the behavioral burden: it discloses determinism/pure-calculation, platform-agnostic operation, primary-metric selection logic, volume floors, noise band, and every refusal code (insufficient_volume, volume_unknown, zero_baseline, no_change, no_primary_metric, evaluation_of_conflicts_with_before, no_comparable_metric). This is unusually thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and every clause carries a real rule, but the single dense paragraph packs a lot of procedural detail that could be structured more scannably. It is not wasteful, just heavy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex, schema-rich, no-annotation, no-output-schema tool, the description covers verdict logic, edge cases, error conditions, and the downstream append path. An agent has everything needed to call it and interpret the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds substantial meaning beyond schema: synonym merging (cost_per_conversion→cpa, cost→spend), days/period interpretation, noise_pct override semantics, and the before/expected refusal interactions with evaluation_of.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource — 'deterministically evaluate whether a logged action's outcome improved, regressed, or is inconclusive' — and scopes it as the reproducible verdict feeding daily-check and /learn. It is clearly distinguishable from sibling tools like mureo_state_action_log_append and mureo_decision_record.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames when to use it ('instead of eyeballing the numbers') and when/how to close an observation via evaluation_of. It also names the sibling to append the result through (mureo_state_action_log_append) and specifies the before-vs-evaluation_of decision rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools