Gumloop MCP Server

Update evaluation config

update_evaluation_config

Destructive

Update an agent's evaluation settings, replacing specified criteria, tags, or data points while preserving omitted fields. Confirmation required.

Instructions

Partially update the evaluation configuration for an agent. Omitted fields keep their current value. Provided list fields (criteria, tags, data_points) replace that list entirely.

Requires Pro tier or above. Explicit confirmation is required for this exact account operation. Runs can spend credits or trigger downstream actions; never resubmit unknown outcomes automatically.

Input Schema

TableJSON Schema

Name	Required	Description
`tags`	No	Tag vocabulary (replaces existing list). Max 50.
`account`	No	Named private Gumloop account; selects private credentials and user/team identity.
`confirm`	No	Set true only when the user asked for exactly this action.
`enabled`	No	Whether evaluations are enabled for this agent.
`payload`	No	Complete JSON request body instead of body flags. Preserves current endpoint fields and values.
`agent_id`	Yes	ID of the agent. Also accepts the reserved aliases `gumball` and `analytics`.
`criteria`	No
`sentiment`	No
`model_name`	No	LLM model to use for evaluation.
`data_points`	No
`payload_file`	No	Regular local JSON body file, at most 5 MB. Cannot be mixed with body flags or payload.
`include_auto_tags`	No	Allow the evaluator to suggest tags beyond your predefined vocabulary.

Schema Changelog

Changes observed during successful MCP inspections.

Changed19 schema fields changedv3.0.0Oct 5, 2026

addedInput schema / $defs / criteria

Added value: +{
+  "description": "Quality criteria to check (replaces existing list). Max 30.",
+  "items": {
+    "properties": {
+      "name": {
+        "type": "string"
+      },
+      "priority": {
+        "enum": [
+          "needs_review",
+          "needs_attention"
+        ],
+        "type": "string"
+      },
+      "prompt": {
+        "description": "True/false statement to evaluate.",
+        "type": "string"
+      },
+      "type": {
+        "enum": [
+          "prohibited_action",
+          "prohibited_words",
+          "voice_tone",
+          "other"
+        ],
+        "type": "string"
+      }
+    },
+    "type": "object"
+  },
+  "type": "array"
+}

addedInput schema / $defs / data_points

Added value: +{
+  "description": "Data points to extract (replaces existing list). Max 40.",
+  "items": {
+    "properties": {
+      "data_type": {
+        "enum": [
+          "string",
+          "boolean",
+          "integer",
+          "number"
+        ],
+        "type": "string"
+      },
+      "description": {
+        "type": "string"
+      },
+      "name": {
+        "type": "string"
+      }
+    },
+    "type": "object"
+  },
+  "type": "array"
+}

changedInput schema / properties / confirm / description

Previous value: -"Must be true for this exact requested account change, agent/flow execution, upload or deletion."New value: +"Set true only when the user asked for exactly this action."

addedInput schema / properties / criteria / $ref
```
Added value: +"#/$defs/criteria"
```

removedInput schema / properties / criteria / description

Removed value: -"Quality criteria to check (replaces existing list). Max 30."

removedInput schema / properties / criteria / items

Removed value: -{
-  "properties": {
-    "name": {
-      "type": "string"
-    },
-    "priority": {
-      "enum": [
-        "needs_review",
-        "needs_attention"
-      ],
-      "type": "string"
-    },
-    "prompt": {
-      "description": "True/false statement to evaluate.",
-      "type": "string"
-    },
-    "type": {
-      "enum": [
-        "prohibited_action",
-        "prohibited_words",
-        "voice_tone",
-        "other"
-      ],
-      "type": "string"
-    }
-  },
-  "type": "object"
-}

removedInput schema / properties / criteria / type
```
Removed value: -"array"
```
addedInput schema / properties / data_points / $ref
```
Added value: +"#/$defs/data_points"
```

removedInput schema / properties / data_points / description

Removed value: -"Data points to extract (replaces existing list). Max 40."

removedInput schema / properties / data_points / items

Removed value: -{
-  "properties": {
-    "data_type": {
-      "enum": [
-        "string",
-        "boolean",
-        "integer",
-        "number"
-      ],
-      "type": "string"
-    },
-    "description": {
-      "type": "string"
-    },
-    "name": {
-      "type": "string"
-    }
-  },
-  "type": "object"
-}

removedInput schema / properties / data_points / type
```
Removed value: -"array"
```
addedInput schema / properties / payload / properties / criteria / $ref
```
Added value: +"#/$defs/criteria"
```

removedInput schema / properties / payload / properties / criteria / description

Removed value: -"Quality criteria to check (replaces existing list). Max 30."

removedInput schema / properties / payload / properties / criteria / items

Removed value: -{
-  "properties": {
-    "name": {
-      "type": "string"
-    },
-    "priority": {
-      "enum": [
-        "needs_review",
-        "needs_attention"
-      ],
-      "type": "string"
-    },
-    "prompt": {
-      "description": "True/false statement to evaluate.",
-      "type": "string"
-    },
-    "type": {
-      "enum": [
-        "prohibited_action",
-        "prohibited_words",
-        "voice_tone",
-        "other"
-      ],
-      "type": "string"
-    }
-  },
-  "type": "object"
-}

removedInput schema / properties / payload / properties / criteria / type
```
Removed value: -"array"
```
addedInput schema / properties / payload / properties / data_points / $ref
```
Added value: +"#/$defs/data_points"
```
removedInput schema / properties / payload / properties / data_points / description
```
Removed value: -"Data points to extract (replaces existing list). Max 40."
```

removedInput schema / properties / payload / properties / data_points / items

Removed value: -{
-  "properties": {
-    "data_type": {
-      "enum": [
-        "string",
-        "boolean",
-        "integer",
-        "number"
-      ],
-      "type": "string"
-    },
-    "description": {
-      "type": "string"
-    },
-    "name": {
-      "type": "string"
-    }
-  },
-  "type": "object"
-}

removedInput schema / properties / payload / properties / data_points / type
```
Removed value: -"array"
```

First observedv2.0.1Oct 3, 2026

TDQS

A4.5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructive=true, idempotent=false and openWorld=true, and the description adds genuinely non-redundant behavior: list fields are wholesale replaced rather than merged, a tier prerequisite gates the call, confirmation must be tied to the exact requested action, and unknown outcomes must not be resubmitted because runs can spend credits. That is exactly the extra context annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the partial-update rule and the list-replacement rule before prerequisites. The closing sentence about credits and downstream actions is somewhat generic to the evaluation domain rather than specific to updating config, so it earns slightly less than a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter, nested, mutation tool with no output schema, the description covers purpose, partial semantics, list replacement, tier gating and confirmation. It leaves the payload/payload_file/body-flag mutual exclusivity and the agent_id aliases to the schema, which is acceptable given 75% coverage, but an agent assembling a call still relies on schema reading.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 12 parameters and 75% schema description coverage, the schema carries most parameter detail, but the description adds the crucial replace-vs-merge semantics for criteria, tags and data_points that the schema does not state. It does not touch payload/payload_file vs body-flag mutual exclusivity, leaving a gap, but the added list semantics are substantive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Partially update the evaluation configuration for an agent') and immediately qualifies the scope with 'partial', which cleanly separates it from the read-side siblings get_evaluation_config / retrieve_evaluation. The verb-plus-resource pairing leaves no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives real operating conditions: omitted fields keep current values, list fields replace entirely, Pro tier is required, and explicit confirmation is required for this exact account operation. It does not name alternative tools (e.g. get_evaluation_config for reading, run_evaluations for executing), so it stops short of explicit when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Update evaluation config

Instructions

Input Schema

Schema Changelog

TDQS

Other Tools