Skip to main content
Glama

ZeroWidth

by ai.zerowidth
OverviewDiscussionsAdmin

Update a Caliper eval (metadata, schedule, alerts)

caliper_evals_update
Destructive

Changes an eval's title, description, visibility, schedule (interval + time anchor), runOnPublish (queue a run whenever the flow publishes), regressionThreshold (score drop vs the previous run that triggers an alert; null = off), or scenarioId (the scenario runs use). Pass only what changes. The dataset, rubric snapshot, and target are fixed at creation — create a new eval to change those. Scheduled and on-publish runs spend credit on their own, so state that plainly when turning them on. May return needs_confirmation.

Input Schema

NameRequiredDescriptionDefault
titleNoNew name.
evalIdYesEval id, from caliper_evals_list.
workspaceNoWorkspace slug. Personal tokens with no default workspace MUST pass this; tokens with a default can override per call. Ignored for workspace API keys.
approvalIdNoApproval id from a prior needs_confirmation response. Omit on the first call.
scenarioIdNoWorkbench-flow evals: the scenario every run uses unless a run picks another (one of the flow's, from workbench_flows_scenarios_list); null clears it.
visibilityNoWho can see it: PRIVATE (only the user), WORKSPACE (every member, the default), or SHARED (specific people, granted afterwards). Say 'make it private' → PRIVATE.
descriptionNoNew description; null clears it.
runOnPublishNoRun whenever the targeted flow publishes a version.
scheduleTimeNoDaily/weekly anchor, HH:MM 24-hour in scheduleTimezone. Null = one interval from now.
scheduleIntervalNoRun cadence; null clears the schedule. Each scheduled run spends credit — say so.
scheduleTimezoneNoIANA timezone for scheduleTime, e.g. America/Chicago.
scheduleDayOfWeekNoWeekly only: 0 = Sunday … 6 = Saturday.
regressionThresholdNoScore drop vs the previous run that raises an alert, on the 0–1 score scale (0.1 = 10 points); null turns alerts off.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / properties / scenarioId
      Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "Workbench-flow evals: the scenario every run uses unless a run picks another (one of the flow's, from workbench_flows_scenarios_list); null clears it."
+}
  2. Changed2 schema fields changed
    • changedInput schema / properties / regressionThreshold / anyOf
      Previous value: -[
-  {
-    "maximum": 5,
-    "minimum": 0.05,
-    "type": "number"
-  },
-  {
-    "type": "null"
-  }
-]New value: +[
+  {
+    "maximum": 1,
+    "minimum": 0.01,
+    "type": "number"
+  },
+  {
+    "type": "null"
+  }
+]
    • changedInput schema / properties / regressionThreshold / description
      Previous value: -"Score drop vs the previous run that raises an alert; null turns alerts off."New value: +"Score drop vs the previous run that raises an alert, on the 0–1 score scale (0.1 = 10 points); null turns alerts off."
  3. First observed

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With destructiveHint=true, the description adds rich context beyond annotations: it explains immutable fields, that scheduled/on-publish runs spend credit, that a 'needs_confirmation' response is possible, and the credit-cost implication of enabling schedulers. This goes well beyond what the annotation flags alone convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, dense paragraph that front-loads the mutable fields, then immutable fields, then a behavioral note. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 13 parameters, no output schema, and annotations indicating destruction. The description covers the update scope, immutability, credit cost, and the needs_confirmation flow, which is nearly complete. It omits a note about the required evalId parameter, but the schema marks it required and describes it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 13 parameters in detail. The description reinforces the most important semantics (e.g., regressionThreshold scale, schedule interval meaning) but does not add new parameter-level information beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific verb (changes/updates) and enumerates the exact mutable fields (title, description, visibility, schedule, runOnPublish, regressionThreshold, scenarioId), which is precise. It does not explicitly distinguish itself from caliper_evals_create, though the phrase 'The dataset, rubric snapshot, and target are fixed at creation — create a new eval to change those' implicitly routes to the create sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear usage guidance: 'Pass only what changes' and delineates immutable fields with an explicit alternative ('create a new eval to change those'). It also states to communicate credit spend when turning on scheduled/on-publish runs. It doesn't distinguish itself from other update-like siblings (e.g., caliper_evals_run), but those are clearly distinct operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All ZeroWidth toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources