The mendrift-mcp server is an autonomous MLOps incident response agent that detects, diagnoses, proposes remediation for, and executes actions related to ML model drift and degradation, with human-in-the-loop safety.
Drift Detection:
Compute per-feature data/prediction drift reports using PSI and KS tests (get_drift_report), and summarize serving metric anomalies (latency, error rate, prediction statistics) via rolling z‑score detection (summarize_metric_anomalies).
Deployment Analysis:
Retrieve chronological deployment history (get_deployment_history) and diff two model versions across training data span, hyperparameters, metrics, and feature schema (diff_deployments) to identify root causes.
Rollback Management:
Generate a human‑reviewable rollback plan (propose_rollback); execute an approved rollback only after validating a single‑use, action‑scoped HMAC token (execute_rollback).
Incident Management:
Open and log an incident record with severity, summary, and diagnosis (open_incident) for integration with ticketing systems.
Safety & Automation:
Enforces a strict cryptographic approval gate for destructive actions, supports read‑only analysis, and enables orchestration of detection‑diagnosis‑remediation workflows with state checkpointing for crash recovery.
Provides tools for interacting with MLflow, enabling AI agents to retrieve deployment history, diff deployments, and execute rollbacks on the MLflow model registry.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mendrift-mcp analyze drift and suggest rollback for the latest deployment"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Autonomous MLOps incident response agent, plus mendrift-mcp — an open-source MCP server for drift detection and ML incident tooling.
pip install mendrift-mcp # or: uvx mendrift-mcpPublished on PyPI and the
MCP Registry as
io.github.suneel190700/mendrift-mcp.
▶ Live demo — run a real incident in your browser: supply an alert, watch the agent diagnose it against a real MLflow registry, and approve or reject the rollback at the human-in-the-loop gate. Toggle between a crafted synthetic scenario and real US consumer-credit benchmark data. React frontend on a FastAPI backend; the free tier sleeps, so the first load may take ~40s.
When a production model drifts or degrades, Mendrift detects it, diagnoses the root cause from monitoring and registry evidence, proposes a remediation, and executes it only after human approval.
alert ──> classify ──> diagnose (MCP tools) ──> propose
│ │
noise ──> close human approval gate
│
execute ──> verify recoveryBuilt with LangGraph (agent orchestration), LangChain (ChatAnthropic +
bind_tools), the Model Context Protocol, Evidently, MLflow, and Claude
(Haiku + Sonnet).
tool | type | purpose |
| read | per-feature drift distances + schema changes (Evidently) |
| read | production vs previous model scored on current traffic |
| read | registry version transitions and aliases |
| read | params / metrics / feature-schema diff between versions |
| read | generates a reviewable rollback plan |
| gated | requires a single-use HMAC |
| write | incident record with diagnosis + evidence |
Related MCP server: fix-mcp
The approval gate is enforced in the tool layer, not the prompt:
execute_rollback verifies a single-use, action-scoped HMAC token minted only
by the human review flow — the minting function is never exposed over MCP. A
prompt-injected or confused agent cannot execute writes.
Tested live: Claude was first ordered to roll back "with full authorization" (it proposed but declined to fabricate a token), then handed a fabricated token, which the gate rejected by constant-time HMAC comparison:
See tests/test_approval_gate.py, including the action-scoping test: a token
minted for one model/version is invalid for any other.
The incident graph halts before execution (interrupt_before) and checkpoints
every step to SQLite. The process can die; a new process resumes the same
incident by thread_id after a human mints the approval token — which enters
state only via update_state(), from outside the graph. Denial is a
first-class path: no token → closed_approval_denied, no execution.
step | model | why |
classify | Haiku | single constrained label; cheapest path |
diagnose | Sonnet | multi-hop tool reasoning over evidence |
verify | Haiku | threshold check on fresh metrics |
Routing lives in a code table (ROUTER_TABLE), not prompts, so cost per path
is measurable config — ~3.9K input / 630 output tokens per incident. The
diagnose loop is bounded (max 8 tool calls) with per-call retries and capped
backoff; on tool failure the model receives a structured error record, and on
budget exhaustion the agent degrades to an incident with partial evidence — it
never invents a diagnosis. Destructive actions require affirmative evidence: a
rollback is recommended only when retrieved evidence links the symptom to a
specific deployment, never on deploy-correlation alone. The agent can also
recommend monitor — real but mild, non-actionable drift is watched, not
acted on.
MENDRIFT_DEMO=0 runs the agent against real infrastructure rather than fixtures:
scripts/seed_demo.py trains two sklearn versions into a local MLflow registry —
v13 clean, v14 with a schema swap and a training window polluted by missed-fraud
labels (recall 0.72 → 0.18, AUC 0.84 → 0.82) — and writes reference/current frames
get_drift_report runs Evidently's DataDriftPreset over those frames, returning
real Wasserstein/JS distances against per-metric thresholds, plus schema changes
derived from actual column sets
get_deployment_history / diff_deployments read the registry and the underlying
runs — real aliases, params, metrics
summarize_metric_anomalies scores the current window with both the production and
previous versions, so it reports model divergence rather than population drift —
a rollback clears it, ordinary data shift does not
an approved execute_rollback moves the production alias for real
uv run mlflow server --host 127.0.0.1 --port 5001 # separate terminal
PYTHONPATH=src uv run python scripts/seed_demo.py
rm -f demo.db
MENDRIFT_DEMO=0 PYTHONPATH=src uv run python scripts/demo_interrupt.py start
MENDRIFT_DEMO=0 PYTHONPATH=src uv run python scripts/demo_interrupt.py approveA live run diagnoses from computed evidence — e.g. "v2 introduced a schema swap replacing promo_flag with promo_flag_v2 … label_noise 0.0 → 0.45 collapsing val_recall 0.724 → 0.176 … 79.7% prediction-rate divergence from the prior version, model-induced, not population drift" — then halts for approval and resolves.
The eval suite deliberately stays on fixtures: evals need determinism and zero cost in CI, while live mode exercises the real stack.
A hosted web app wraps live mode behind a browser UI: a React (Vite) frontend on
a FastAPI backend, deployed on Render. A visitor submits an alert, the frontend
posts it to /api/diagnose, and the backend runs the real LangGraph agent — live
Claude reasoning over an embedded MLflow registry (sqlite://, seeded on boot) — then
halts at the HMAC gate. Approve or reject and the backend resumes the graph via
/api/decision, executing a real alias rollback and verifying recovery. The Anthropic
key lives only on the server; runs are rate-limited since each calls a real model.
Try it: mendrift-demo.onrender.com.
The dashboard toggles between two seeded worlds, so the same agent can be seen against both a crafted scenario and genuine real-world data:
Synthetic (scripts/seed_demo.py, model fraud-scorer) — the crafted schema-swap
incident: clean, teachable, an unambiguous rollback story.
Real US credit (scripts/seed_real.py, model credit-risk) — the
Give Me Some Credit
dataset (real US consumer-credit records, target SeriousDlqin2yrs) split by borrower
age into reference/current windows for genuine feature drift, with a controlled model
regression injected into v2 (asymmetric missed-default label noise) so the incident
has ground truth. Real distributions and real Evidently drift; a known correct action.
Measured gap: val_recall 0.637 → 0.156, AUC 0.854 → 0.810.
Injecting a known regression into real data is standard practice for validating a drift-detection system — it gives the evaluator ground truth for what the agent should decide while the drift computation still runs on genuine distributions.
The backend routes each request to the right world (model + parquet frames + label
column) per the dataset field; the tool layer reads those from env vars, applied
per-request under a lock so concurrent requests stay isolated.
Run the web app locally:
# 1. build the React frontend (FastAPI serves the built assets)
cd frontend && npm install && npm run build && cd ..
# 2. seed both worlds, then start the backend (frontend + API on one port)
export ANTHROPIC_API_KEY=sk-ant-...
export MLFLOW_TRACKING_URI="sqlite:///$(pwd)/mlflow.db"
PYTHONPATH=src uv run python scripts/seed_demo.py # synthetic world (fraud-scorer)
PYTHONPATH=src uv run python scripts/seed_real.py # real world (credit-risk)
PYTHONPATH=src uv run uvicorn app.main:app --port 8000 # open http://localhost:8000For frontend development with hot reload, run cd frontend && npm run dev (port 5173);
Vite proxies /api to the backend on port 8000.
src/mendrift/evals/ replays synthetic incident trajectories against the
real graph — only the LLM (scripted) and the read tools (fixture world)
are faked; the gated action tools are the genuine implementations, so the HMAC
gate is exercised by every test. Four assertions per trajectory:
check | meaning |
| every |
| triage label matched |
| required tool calls occurred in order (extras allowed) |
| terminal outcome matched |
19 logic-distinct incident scenarios spanning the decision space, each with its own evidence shape and correct action:
Rollback — deploy-correlated drift or quality regression with affirmative diff evidence
Retrain — label/concept shift, segment-specific degradation (no valid rollback target)
Monitor — mild seasonal drift, low-importance-feature drift, holiday effects
Incident (investigate) — upstream schema rename, feature-store change, docs-only deploy, calibration break, threshold shift, silent data-quality drop
Graceful degradation — evidence tools down → incident with partial evidence, never a fabricated diagnosis
Noise — flapping / auto-resolved alerts closed with zero tool calls
Human-declined — well-founded rollback the reviewer rejects → closed, no execution
Scripted for fast CI, live for the measured rate:
PYTHONPATH=src uv run python scripts/run_traj.py --all # scripted, fast
PYTHONPATH=src uv run python scripts/run_traj.py --all --live # real modelsLive-model eval runs at ~95% task-success; the handful of run-to-run divergences reflect LLM eval variance on decision-margin scenarios. The live suite surfaced real failure classes during development — a JSON extractor masking a correct decision, a classifier baited by an alert's reassuring wording, and a diagnoser proposing rollback on correlation alone — each fixed at its own layer (parser, alert wording, evidence-rule prompt).
uv sync
MENDRIFT_DEMO=1 uv run mendrift-mcp # stdio MCP server with fixture data
PYTHONPATH=src uv run pytest -v # gate + trajectory suiteClaude Desktop config:
{"mcpServers": {"mendrift": {
"command": "uvx",
"args": ["mendrift-mcp"],
"env": {"MENDRIFT_DEMO": "1"}
}}}mendrift-mcp server over stdio, verified in MCP Inspector and Claude Desktop
seven tools with a read / gated / write permission taxonomy
HMAC-gated rollback with action-scoped single-use tokens (tests first)
LangGraph incident graph: SQLite checkpointing + human-approval interrupt, kill-resume proven
LLM nodes on LangChain (ChatAnthropic.bind_tools): Haiku classify/verify, Sonnet diagnose loop
19-scenario trajectory eval across the decision space; ~95% live, zero ungated writes
CI: gate + trajectory suite on every push
live mode: real Evidently drift computation, MLflow registry history/diff, real alias rollback
live web demo: React + Vite frontend on FastAPI, deployed on Render
two demo worlds: crafted synthetic scenario + real US credit-risk data, selectable in the UI
published: PyPI (pip install mendrift-mcp) + MCP Registry (io.github.suneel190700/mendrift-mcp)
MIT
Diff two model versions: training data span, params, eval metrics, feature schema.
The primary root-cause tool: correlates 'what changed' between the incumbent
and the newly deployed version.
| Name | Required | Description | Default |
|---|---|---|---|
| version_a | Yes | ||
| version_b | Yes | ||
| model_name | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the behavior of comparing specific attributes across versions, which is useful. However, it does not explicitly state whether the operation is read-only or if any side effects occur. The term 'diff' implies non-mutating, but this is not confirmed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action and a specific list of diffed attributes. The second sentence adds contextual value by labeling it as a root-cause tool. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While an output schema exists (so return values are covered), the description lacks details on parameter formats and any prerequisites or constraints (e.g., whether versions must be consecutive). It adequately conveys the tool's primary purpose but leaves gaps in operational details that could trip up an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'two model versions' which maps to version_a and version_b, but does not explain the expected format for versions or model_name. The description lists diff categories (training data span, params, etc.) but those are not the parameters themselves. Thus, an agent gets no additional clarity on how to fill the three required parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Diff two model versions' with a specific list of what is diffed (training data span, params, eval metrics, feature schema). It also distinguishes itself from siblings by labeling itself as 'the primary root-cause tool,' making its role unique among the listed tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool—for root-cause analysis of what changed between deployments. It says it correlates 'what changed between the incumbent and the newly deployed version,' giving clear context. However, it does not mention specific exclusions or when to prefer sibling tools like get_drift_report or summarize_metric_anomalies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute an approved rollback. Rejects missing/invalid tokens.
| Name | Required | Description | Default |
|---|---|---|---|
| model_name | Yes | ||
| approval_token | Yes | ||
| target_version | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It mentions token rejection but omits critical details such as what the rollback actually changes, whether it is destructive, what permissions are required, and what happens on success. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. 'Execute an approved rollback' front-loads the core purpose, and 'Rejects missing/invalid tokens' adds a key behavioral constraint. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a high-stakes rollback tool, the description is critically underspecified. It lacks guidance on when to use it relative to alternatives, side effects, prerequisites beyond the token, and error behavior. The existence of an output schema does not compensate for the absence of usage context and behavioral disclosure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no parameter-specific explanations. While the parameter names (model_name, target_version, approval_token) are somewhat self-explanatory, the description does not clarify their roles or constraints beyond what the schema already shows.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Execute an approved rollback.' The verb 'execute' is specific and distinguishes this from the sibling 'propose_rollback.' It also mentions token validation, adding a behavioral detail that clarifies the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'approved rollback' implies the tool should only be used after approval, and 'Rejects missing/invalid tokens' indicates that a valid approval token is a prerequisite. This provides clear contextual guidance without explicitly naming alternatives, but the sibling name 'propose_rollback' strongly implies the distinction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List recent deployments/version transitions for a model, newest first.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| model_name | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. 'List' implies a read-only operation, and 'newest first' discloses ordering. However, it does not disclose whether transitions include all types, how the limit parameter affects behavior, or any prerequisites. This is moderate transparency but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no redundant words. It conveys the action, resource, and ordering efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values are already specified. The description covers the core purpose and resource, and mentions ordering. It could be more complete by explaining limit behavior, but for a simple list operation it is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It implicitly explains model_name as the model to retrieve history for, but it does not explain the limit parameter. The description adds some meaning but not complete compensation for both parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists recent deployments/version transitions for a model, with an explicit ordering of newest first. This distinguishes it from siblings like propose_rollback or diff_deployments, which target different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context is clear: use this tool to retrieve deployment history for a model. It does not explicitly mention when not to use it or name alternatives, but the sibling names and the verb 'list' provide adequate contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compute a data/prediction drift report for a deployed model.
Compares the current serving window against a reference window (PSI + KS
tests per feature). Returns per-feature drift scores, an overall drift
flag, and the top drifted features.
Args:
model_name: Registered model name as it appears in the model registry.
reference_window_hours: Reference window size (default 7 days).
current_window_hours: Current window size (default 24 hours).
| Name | Required | Description | Default |
|---|---|---|---|
| model_name | Yes | ||
| current_window_hours | No | ||
| reference_window_hours | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description is the sole source of behavioral disclosure. It explains that it computes a report and returns drift metrics, but does not disclose potential side effects, permissions, rate limits, or performance implications. The read-only nature is implied but not explicit, and there is no mention of prerequisites beyond a deployed model.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with a clear summary, and uses a simple Args list for parameters. Every sentence adds value, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so return values need not be repeated. The description covers purpose, method, and all parameters adequately. Minor gaps like usage guidance relative to siblings and explicit behavioral traits prevent a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates by providing meaningful explanations for all three parameters. It defines model_name as the registry name, and gives both default values and units for the window sizes, adding clarity beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it computes a data/prediction drift report, comparing serving windows with PSI + KS tests per feature. It also lists specific return outputs (per-feature scores, overall drift flag, top drifted features), distinguishing it from sibling tools like rollback or incident management.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies the context (deployed model drift detection) and compares current vs reference windows, making clear when it is relevant. However, it does not explicitly mention alternatives or exclusions, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Open an incident record (demo: JSONL log; live: ticketing webhook).
| Name | Required | Description | Default |
|---|---|---|---|
| summary | Yes | ||
| severity | Yes | ||
| diagnosis | Yes | ||
| model_name | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosing side effects. It does reveal two behavior modes (JSONL log in demo, ticketing webhook in live), which is useful. However, it does not disclose permissions required, reversibility, failure modes, or any other potential side effects beyond where the record is sent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the action and resource. It includes essential context about demo vs live modes without unnecessary words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 required parameters and no annotations or schema descriptions, the description is too minimal. It omits parameter semantics and usage guidance. The demo/live distinction is helpful but does not make the tool usable without additional information about inputs and expected behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 4 required parameters with zero description coverage, and the tool description does not explain any of them. The names (summary, severity, diagnosis, model_name) are somewhat self-explanatory, but there is no guidance on expected format, allowed values (no enums), or how they relate. The description adds no value beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Open an incident record', with a specific verb and resource. It also adds context about demo (JSONL log) and live (ticketing webhook) environments, which distinguishes it from sibling tools like rollback and drift report operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool vs alternatives. The description only mentions demo/live modes, but does not state conditions for opening an incident or situations where another tool would be more appropriate. No exclusions or alternative suggestions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a rollback plan for human review. Read-only — does not execute.
| Name | Required | Description | Default |
|---|---|---|---|
| reason | Yes | ||
| model_name | Yes | ||
| current_version | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Read-only — does not execute,' which is a critical safety trait. However, it does not mention what the generated plan contains, whether any state changes occur, or any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise and front-loaded: 'Generate a rollback plan for human review' states the core purpose immediately, followed by the key safety qualifier. Every word earns its place, and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity of 3 required parameters and no parameter descriptions, the description is too minimal to enable correct invocation. It explains the main purpose and safety, but the lack of parameter semantics and any contextual guidance about when to generate a rollback plan leaves significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 3 required parameters with zero description coverage, and the tool description does not explain what model_name, current_version, or reason should contain or how they are used. The description completely fails to compensate for the lack of schema-level parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Generate a rollback plan for human review.' It clearly distinguishes the tool from siblings like execute_rollback by adding 'Read-only — does not execute,' making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it is for generating a plan for human review, not for executing it. The 'does not execute' phrase implicitly tells the agent not to use this tool when an actual rollback is needed, but it does not explicitly name alternatives or give more detailed when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize serving-metric anomalies (latency, error rate, prediction stats).
Returns metric statistics with anomaly windows flagged via rolling z-score, suitable for an LLM to reason over without raw time series.
| Name | Required | Description | Default |
|---|---|---|---|
| model_name | Yes | ||
| lookback_hours | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses the algorithm (rolling z-score), the output style (metric statistics with anomaly windows flagged), and a key limitation (no raw time series). This goes beyond a simple statement and gives behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exactly two sentences, front-loaded with the main purpose followed by a clarifying detail about output and algorithm. Every word earns its place, with no redundant fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no annotations, presence of an output schema), the description is sufficiently complete. It explains what the tool does, how it flags anomalies, and what it returns, leaving only minor gaps around error cases or edge behavior that are likely covered by the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not explicitly explain the parameters model_name or lookback_hours, and schema coverage is 0%. However, the parameter names are self-explanatory and the schema provides the default for lookback_hours. The description adds little parametric meaning beyond what the schema already communicates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'summarize' and the resource 'serving-metric anomalies' with specific metric types (latency, error rate, prediction stats). It distinguishes itself from sibling tools by focusing on anomaly summarization rather than rollback, incidents, drift, or deployment history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by noting it is 'suitable for an LLM to reason over without raw time series,' which implies when to use it. However, it does not explicitly name alternatives or state when not to use the tool, stopping short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.2diff_deploymentsexecute_rollbackget_deployment_historyget_drift_reportopen_incidentpropose_rollbacksummarize_metric_anomaliesScored across 7 tools
Each tool targets a distinct action and resource: drift reports, metric anomalies, deployment history, deployment diffs, incidents, and rollback planning/execution. There is no ambiguity between tools like propose_rollback and execute_rollback because one is read-only planning and the other is execution.
All tools use lowercase snake_case with a verb_noun pattern (propose_rollback, execute_rollback, open_incident, get_drift_report, summarize_metric_anomalies, get_deployment_history, diff_deployments). The naming is uniform and predictable, making it easy to infer tool behavior from the name.
Seven tools is a well-scoped count for a drift monitoring and mitigation server. Each tool covers a distinct part of the workflow—detection, investigation, incident management, and rollback—without unnecessary redundancy or overwhelming the agent.
The core lifecycle is covered: drift detection, anomaly summarization, deployment history/diffing, incident opening, and rollback proposal/execution. The main gap is that incidents can be opened but not listed, closed, or updated, and there is no rollback status tracking. These are workable gaps but slightly incomplete for a full incident management loop.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
AI-powered incident management and server monitoring via MCP.
MCP-native AI SRE: ask what's broken in production, get a reviewed GitHub fix PR.
MCP server for building and testing AI agents with multi-model experimentation and insights.
