Skip to main content
Glama

mcp-regression-watch

MCP server for the Agent Update Regression Watch - deploy-triggered regression testing: post-update behavior-diffing against baselines, drift alerts, retained regression-comparison reports. Scheduled benchmarks catch planned drift; this catches the unplanned kind.

Pricing: $29-99/mo (rides the proofspan tier).

Tools

run_regression_check

Run a deploy-triggered regression check against the persisted baseline.

Parameter

Type

Default

Description

service

string

required

Service being checked (e.g. mcp-agent-sla)

version

string

required

Version under test (e.g. v2.1.0)

seed

integer

-

Optional seed for reproducible simulation

persist

boolean

true

Persist the run to state

Returns per-case results with baseline value, current value, delta, pct drift, threshold, and status. Overall pass/fail plus breach list. Persists the raw run log (customer-owned exit material per two-column IP rule).

get_drift_report

Aggregate persisted run history into a drift report.

Parameter

Type

Default

Description

service

string

required

Service to report on

window_days

integer

30

Look-back window in days (1-365)

Returns drift trends per metric over time, breach counts, worst-offending version, drift heat summary, and alert list. All values computed from actual stored runs via Python statistics.

get_comparison_report

Retained comparison report between two versions.

Parameter

Type

Default

Description

service

string

required

Service to compare

from_version

string

required

Baseline version

to_version

string

required

Candidate version

Returns per-case side-by-side table, diff summary (improved/regressed/unchanged/added/removed), severity-ranked regression list, and content hash.

set_drift_threshold

Persist a per-metric drift threshold.

Parameter

Type

Default

Description

metric

string enum

required

Metric (latency_p95_ms, tool_call_accuracy, hallucination_rate, json_schema_validity, token_cost, refusal_precision, retry_rate, throughput_rps, error_rate)

threshold_pct

number

required

Drift threshold as percentage of baseline (0.1-100.0)

persist

boolean

true

Persist to state

Related MCP server: deploy-check

Installation

pip install -r requirements.txt

Usage

python main.py

State persisted in ./state/regression_state.json. Override with MCP_REGRESSION_WATCH_STATE_DIR.

Connecting to an MCP client

{
  "mcpServers": {
    "regression-watch": {
      "command": "python3",
      "args": ["/tmp/v3-services/mcp-regression-watch/main.py"]
    }
  }
}

License

MIT

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to prove that previously working behavior still works after code changes by running the product through the same steps twice and surfacing only unintended differences as actionable findings.
      19 npm
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      Enables agents to verify that a deploy actually worked by checking pages, links, and comparing environments, catching blank 200 responses, broken links, stray noindex, and staging that never updated.
      3
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables ingesting caller-supplied agent execution traces and comparing them against signed, versioned baselines of normal behaviour to surface silent drift. Reports severity with per-dimension attribution, a replayable exemplar episode, and visible suppression of declared maintenance windows or known change markers.
      1
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables MCP clients to check whether third-party API responses have drifted, inspect expected schemas, review change history, list monitored endpoints, and trigger checks before agents trust an API.
      278 PyPI
      1
      MIT