Skip to main content
Glama
Sign In
iris-eval/mcp-server

iris-eval/mcp-server

by iris-eval
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Hybrid

Deploy Custom Rule

deploy_rule

Deploy a custom evaluation rule that persists and activates immediately, firing on every future evaluate_output call for its bundle with audited severity-based scoring.

Instructions

Deploy a custom rule that fires on every future evaluate_output call of its bundle — persisted, active immediately, audited.

What it does. Stores the rule and loads it into the running engine. eval_type says WHEN it fires (that bundle, and eval_type="all"); severity says what a failure does: low and medium lower the score, high and critical fail the verdict. definition.type picks the check, definition.config its keys. A taken name is refused unless replace is true. Full detail: iris://capabilities.

When not to use it. For a one-off check (custom_rules on evaluate_output). To pause a rule (delete_rule with enabled: false).

Returns. JSON: rule, replaced, warning.

Errors. IRIS_DUPLICATE_RULE; IRIS_INVALID_RULE_CONFIG, naming the field; IRIS_STORAGE_ERROR. Failures return {"error":{code,message,recovery}} with isError; follow recovery.

Siblings. list_rules — see what is deployed; delete_rule — remove or disable; evaluate_output — where it fires.

Input Schema

NameRequiredDescriptionDefault
nameYesRule name, 1-80 letters, digits, dots, dashes and underscores (used in eval results). Must be unique among deployed rules unless replace=true
replaceNotrue replaces a deployed rule of the same name (fresh id, audit kept); false (default) refuses
evalTypeNocamelCase alias of eval_type, accepted for compatibility — prefer eval_type (snake_case is canonical across the tools)
severityNoWhat a FAILURE of this rule means. low/medium: informational — contributes to the weighted score only (plus dashboard sort + audit alerts). high/critical: hard-fail — a failing evaluation of this rule forces the overall passed=false regardless of the weighted scoremedium
eval_typeNoThe bundle this rule fires in (and on eval_type="all"); pass one of eval_type / evalType
definitionYesThe check: type, config and optional weight; an unknown key is rejected
descriptionNoWhat this rule checks for and why it matters
sourceMomentIdNocamelCase alias of source_moment_id, accepted for compatibility — prefer source_moment_id
source_moment_idNoOptional id of the Decision Moment this rule came from; pass one of source_moment_id / sourceMomentId

Output Schema

NameRequiredDescriptionDefault
ruleYes
warningNo
replacedNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed19 schema fields changedv0.10.1
    • changedInput schema / properties / definition / description
      Previous value: -"Check definition (regex, length, keyword, cost, or schema). Accepts exactly type, config, weight and an optional name — an unknown key is rejected"New value: +"The check: type, config and optional weight; an unknown key is rejected"
    • changedInput schema / properties / definition / properties / config / description
      Previous value: -"Check configuration; required keys depend on type (regex_match: pattern; min_length: min_length; max_length: max_length; contains_keywords/excludes_keywords: keywords; cost_threshold: max_cost; json_schema: none)"New value: +"Check configuration; required keys depend on type (regex_match: pattern; min_length: min_length; max_length: max_length; contains_keywords/excludes_keywords: keywords; cost_threshold: max_cost; json_schema: schema, optional; action_policy: allow and/or deny)"
    • changedInput schema / properties / definition / properties / name / description
      Previous value: -"Optional and IGNORED if given — the server overwrites it with the top-level `name` so the rule reports under one name everywhere"New value: +"Ignored: the server overwrites it with the top-level `name`"
    • changedInput schema / properties / definition / properties / type / enum
      Previous value: -[
-  "regex_match",
-  "regex_no_match",
-  "min_length",
-  "max_length",
-  "contains_keywords",
-  "excludes_keywords",
-  "json_schema",
-  "cost_threshold"
-]New value: +[
+  "regex_match",
+  "regex_no_match",
+  "min_length",
+  "max_length",
+  "contains_keywords",
+  "excludes_keywords",
+  "json_schema",
+  "cost_threshold",
+  "action_policy"
+]
    • changedInput schema / properties / eval_type / description
      Previous value: -"Eval category this rule belongs to; the rule fires on evaluate_output calls whose eval_type equals it (and on eval_type=\"all\"). Canonical snake_case spelling — pass exactly one of eval_type / evalType"New value: +"The bundle this rule fires in (and on eval_type=\"all\"); pass one of eval_type / evalType"
    • changedInput schema / properties / name / description
      Previous value: -"Human-readable rule name (1-80 chars; used in eval results). Must be unique among deployed rules unless replace=true"New value: +"Rule name, 1-80 letters, digits, dots, dashes and underscores (used in eval results). Must be unique among deployed rules unless replace=true"
    • addedInput schema / properties / name / pattern
      Added value: +"^[a-z0-9._-]+$"
    • changedInput schema / properties / replace / description
      Previous value: -"When a rule with this name is already deployed: false (default) rejects the call; true deletes the existing same-named rule(s) and deploys this one in their place (fresh id; audit rows preserved)"New value: +"true replaces a deployed rule of the same name (fresh id, audit kept); false (default) refuses"
    • changedInput schema / properties / source_moment_id / description
      Previous value: -"Optional Decision Moment ID the rule was derived from (preserves workflow-inversion provenance). Canonical snake_case — pass exactly one of source_moment_id / sourceMomentId"New value: +"Optional id of the Decision Moment this rule came from; pass one of source_moment_id / sourceMomentId"
    • removedOutput schema / properties / replaced / description
      Removed value: -"with replace: true, the earlier rule(s) of the same name that were retired"
    • removedOutput schema / properties / replaced / items / additionalProperties
      Removed value: -{}
    • removedOutput schema / properties / replaced / items / properties
      Removed value: -{
-  "id": {
-    "type": "string"
-  }
-}
    • removedOutput schema / properties / replaced / items / required
      Removed value: -[
-  "id"
-]
    • removedOutput schema / properties / replaced / items / type
      Removed value: -"object"
    • removedOutput schema / properties / rule / description
      Removed value: -"the rule as persisted: id (rule-<hex>, keep it for delete_rule), name, description, evalType, severity, definition, enabled, createdAt, updatedAt, version, sourceMomentId"
    • removedOutput schema / properties / rule / properties / id
      Removed value: -{
-  "type": "string"
-}
    • removedOutput schema / properties / rule / properties / name
      Removed value: -{
-  "type": "string"
-}
    • removedOutput schema / properties / rule / required
      Removed value: -[
-  "id",
-  "name"
-]
    • removedOutput schema / properties / warning / description
      Removed value: -"with replace: true, one sentence naming what was retired"
  2. Changed1 schema field changedv0.9.0
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "additionalProperties": {},
+  "properties": {
+    "replaced": {
+      "description": "with replace: true, the earlier rule(s) of the same name that were retired",
+      "items": {
+        "additionalProperties": {},
+        "properties": {
+          "id": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "id"
+        ],
+        "type": "object"
+      },
+      "type": "array"
+    },
+    "rule": {
+      "additionalProperties": {},
+      "description": "the rule as persisted: id (rule-<hex>, keep it for delete_rule), name, description, evalType, severity, definition, enabled, createdAt, updatedAt, version, sourceMomentId",
+      "properties": {
+        "id": {
+          "type": "string"
+        },
+        "name": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "id",
+        "name"
+      ],
+      "type": "object"
+    },
+    "warning": {
+      "description": "with replace: true, one sentence naming what was retired",
+      "type": "string"
+    }
+  },
+  "required": [
+    "rule"
+  ],
+  "type": "object"
+}
  3. Changed20 schema fields changedv0.8.0
    • addedInput schema / additionalProperties
      Added value: +false
    • addedInput schema / properties / definition / additionalProperties
      Added value: +false
    • changedInput schema / properties / definition / description
      Previous value: -"Check definition (regex, length, keyword, cost, or schema)"New value: +"Check definition (regex, length, keyword, cost, or schema). Accepts exactly type, config, weight and an optional name — an unknown key is rejected"
    • addedInput schema / properties / definition / properties / config / description
      Added value: +"Check configuration; required keys depend on type (regex_match: pattern; min_length: min_length; max_length: max_length; contains_keywords/excludes_keywords: keywords; cost_threshold: max_cost; json_schema: none)"
    • addedInput schema / properties / definition / properties / name / description
      Added value: +"Optional and IGNORED if given — the server overwrites it with the top-level `name` so the rule reports under one name everywhere"
    • addedInput schema / properties / definition / properties / name / maxLength
      Added value: +80
    • addedInput schema / properties / definition / properties / name / minLength
      Added value: +1
    • addedInput schema / properties / definition / properties / type / description
      Added value: +"Check type — decides which config keys are required"
    • addedInput schema / properties / definition / properties / weight / description
      Added value: +"Weight in the weighted score (default 1; must be > 0)"
    • addedInput schema / properties / definition / properties / weight / exclusiveMinimum
      Added value: +0
    • changedInput schema / properties / definition / required
      Previous value: -[
-  "name",
-  "type",
-  "config"
-]New value: +[
+  "type",
+  "config"
+]
    • changedInput schema / properties / evalType / description
      Previous value: -"Eval category this rule belongs to; determines when it fires"New value: +"camelCase alias of eval_type, accepted for compatibility — prefer eval_type (snake_case is canonical across the tools)"
    • addedInput schema / properties / eval_type
      Added value: +{
+  "description": "Eval category this rule belongs to; the rule fires on evaluate_output calls whose eval_type equals it (and on eval_type=\"all\"). Canonical snake_case spelling — pass exactly one of eval_type / evalType",
+  "enum": [
+    "completeness",
+    "relevance",
+    "safety",
+    "cost",
+    "custom"
+  ],
+  "type": "string"
+}
    • changedInput schema / properties / name / description
      Previous value: -"Human-readable rule name (used in eval results)"New value: +"Human-readable rule name (1-80 chars; used in eval results). Must be unique among deployed rules unless replace=true"
    • changedInput schema / properties / name / maxLength
      Previous value: -120New value: +80
    • addedInput schema / properties / replace
      Added value: +{
+  "default": false,
+  "description": "When a rule with this name is already deployed: false (default) rejects the call; true deletes the existing same-named rule(s) and deploys this one in their place (fresh id; audit rows preserved)",
+  "type": "boolean"
+}
    • changedInput schema / properties / severity / description
      Previous value: -"Severity used for dashboard sort + audit alerts"New value: +"What a FAILURE of this rule means. low/medium: informational — contributes to the weighted score only (plus dashboard sort + audit alerts). high/critical: hard-fail — a failing evaluation of this rule forces the overall passed=false regardless of the weighted score"
    • changedInput schema / properties / sourceMomentId / description
      Previous value: -"Optional Decision Moment ID the rule was derived from (preserves workflow-inversion provenance)"New value: +"camelCase alias of source_moment_id, accepted for compatibility — prefer source_moment_id"
    • addedInput schema / properties / source_moment_id
      Added value: +{
+  "description": "Optional Decision Moment ID the rule was derived from (preserves workflow-inversion provenance). Canonical snake_case — pass exactly one of source_moment_id / sourceMomentId",
+  "type": "string"
+}
    • changedInput schema / required
      Previous value: -[
-  "name",
-  "evalType",
-  "definition"
-]New value: +[
+  "name",
+  "definition"
+]
  4. Changed3 schema fields changedv0.4.6
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / definition / additionalProperties
      Removed value: -false
    • addedInput schema / properties / definition / properties / config / propertyNames
      Added value: +{
+  "type": "string"
+}
  5. Addedv0.1.10

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false, destructiveHint=false, and idempotentHint=false, but the description adds substantial value beyond them: the rule is persisted, active immediately, and audited; a taken name is refused unless replace is true; and the exact error codes (IRIS_DUPLICATE_RULE, IRIS_INVALID_RULE_CONFIG, IRIS_STORAGE_ERROR) plus the {error:{code,message,recovery}} envelope are disclosed. This is unusually complete behavioral disclosure for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core behavior in the opening sentence, then segmented into 'What it does', 'When not to use it', 'Returns', 'Errors', and 'Siblings'. Despite its length, every section conveys actionable information with no filler, and an agent can skim the labeled sections quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even though an output schema exists, the 'Returns' section names the top-level keys (rule, replaced, warning), and the 'Errors' section covers failure recovery. Combined with sibling routing and detailed mutation semantics, the description provides everything an agent needs to invoke deploy_rule correctly and handle outcomes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline would be 3, and the description pushes past it by explaining the causal semantics of eval_type (WHEN it fires) and severity (low/medium lower score, high/critical hard-fail) in prose rather than merely restating enums. It stops short of documenting specific config keys per type, but the schema already does that richly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb (deploy), resource (custom rule), and effect (fires on every future evaluate_output call of its bundle, persisted, active immediately, audited). A 'What it does' section further pins down eval_type, severity, and definition.type semantics, so an agent can distinguish it from siblings without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is an explicit 'When not to use it' section with alternatives named: use custom_rules on evaluate_output for a one-off check, and delete_rule with enabled: false to pause. The 'Siblings' section routes list_rules, delete_rule, and evaluate_output to their purposes, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools