Skip to main content
Glama
Sign In
SAS MCP Server

SAS MCP Server

Official
by sassoftware
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

Set Kpi Alert Rule

set_kpi_alert_rule
DestructiveIdempotent

Configure KPI alert rules on a model project to trigger alerts when monitored accuracy, drift, or fairness metrics cross defined thresholds.

Instructions

Set a KPI alert rule on a model project: alert when a monitoring KPI crosses a threshold.

Model Manager checks the project's models against its rules whenever their KPIs are calculated (run_performance_definition) and lists them as the project's KPI rules. kpi is one of:

  • accuracy: AUC, misclassification, TPR, FPR, F1, ASE, RASE, R-squared, global accuracy, cumulative lift _Cumlift5_ to _Cumlift30_, or the KPI names get_project_kpis lists;

  • drift: devIndex (the deviation index) of each variable in variables, input or output — one rule per variable;

  • fairness: EqualOpportunity, DemographicParity, PredictiveParity, EqualAccuracy or EqualizedOdds for the sensitive variable in variables.

operator is greaterThan (alert above the threshold, as for misclassification or drift) or lessThan (alert below it, as for AUC). A rule for the same KPI, variable and direction is updated, not duplicated; alert_rule_id updates a given rule, its direction too. The result says whether the champion breaks the rule now.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "project_id": "<project_id>",
      "kpi": "<kpi>",
      "operator": "<operator>",
      "threshold": "<threshold>"
    },
    "name": "set_kpi_alert_rule"
  }
}

Input Schema

NameRequiredDescriptionDefault
kpiYesThe KPI to watch.
nameNoThe rule's name (default as Model Manager names it, "AUC Less than 0.9"); ``{variable}`` is replaced by each variable.
operatorYes``greaterThan`` or ``lessThan``.
severityNo``low``, ``medium`` (default) or ``high``: Model Manager's priority.medium
thresholdYesThe value the KPI should not cross.
variablesNoThe variables of a drift or fairness KPI.
project_idYesThe project.
occurrencesNoAlert after this many occurrences (default 1).
alert_rule_idNoA rule to update instead.

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.19.2

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (destructiveHint=true, idempotentHint=true), the description concretely explains the upsert identity key ('same KPI, variable and direction is updated, not duplicated') and that alert_rule_id targets a given rule including its direction. It also discloses the return semantics ('whether the champion breaks the rule now') and the multi-rule expansion behavior for drift/fairness variables, which is exactly the kind of surprise an agent needs warned about.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with a one-sentence summary, then a structured bullet list of KPI families, then operator and update semantics. The content is dense but nearly all of it is load-bearing given there are no enum constraints; only the RST double-backtick formatting and some long enumerations slightly reduce readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutation tool with an output schema and annotations already covering the safety profile, the description supplies the missing pieces: valid KPI names per family, operator direction, variable handling, update/duplication semantics, and evaluation timing. Nothing an agent needs in order to construct a correct call is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage the baseline would be 3, but the description adds meaning the schema cannot: it enumerates valid kpi values per family (accuracy, drift, fairness), explains that drift produces one rule per variable in `variables` and fairness uses the sensitive variable there, and defines operator direction with concrete examples (greaterThan for misclassification/drift, lessThan for AUC).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a specific verb and resource ('Set a KPI alert rule on a model project') plus the actual effect ('alert when a monitoring KPI crosses a threshold'). It is clearly distinct from siblings such as get_project_kpis, run_performance_definition, and delete_kpi_alert_rule without needing any schema inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description situates the tool in a workflow by explaining that Model Manager checks models against these rules when KPIs are calculated via run_performance_definition, and it tells the agent that existing rules are updated rather than duplicated. It stops short of explicitly naming alternatives ('to remove a rule use delete_kpi_alert_rule', 'to inspect current rules use get_project_kpis'), so routing guidance is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools