Replai

Edit test case

test_case_edit

DestructiveIdempotent

Edits supplied question/rubric; question changes invalidate old answers and grades.

No revision CAS exists. test_case_read shows state after response loss.

Input Schema

Name	Required	Description
`data`	Yes	Changes to the saved test question and required or advisory conditions.
`case_id`	Yes	Saved test case ID from test_cases_list or test_case_create.
`agent_id`	Yes	Agent ID from agents_list or agent_create.

Output Schema

Name	Required	Description	Default
`id`	Yes
`origin`	No
`question`	Yes
`created_at`	Yes
`evaluation`	No
`last_run_at`	No
`draft_answer`	No
`expectations`	No
`last_attempt`	No
`draft_revision`	No
`answer_freshness`	No
`published_answer`	No
`freshness_reasons`	No
`published_version`	No
`evaluation_current`	No
`evaluation_criteria`	No
`advisory_expectations`	No
`draft_source_versions`	No
`image_evidence_available`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed3 schema fields changedOct 30, 2026

addedInput schema / properties / agent_id / description

Added value: +"Agent ID from agents_list or agent_create."

addedInput schema / properties / case_id / description

Added value: +"Saved test case ID from test_cases_list or test_case_create."

addedInput schema / properties / data / description

Added value: +"Changes to the saved test question and required or advisory conditions."

Changed3 schema fields changedOct 30, 2026

addedOutput schema / $defs / FreshnessReason

Added value: +{
+  "properties": {
+    "kind": {
+      "enum": [
+        "unrecorded",
+        "settings",
+        "source_added",
+        "source_removed",
+        "source_changed"
+      ],
+      "title": "Kind",
+      "type": "string"
+    },
+    "source_id": {
+      "anyOf": [
+        {
+          "type": "string"
+        },
+        {
+          "type": "null"
+        }
+      ],
+      "default": null,
+      "title": "Source Id"
+    },
+    "source_name": {
+      "anyOf": [
+        {
+          "type": "string"
+        },
+        {
+          "type": "null"
+        }
+      ],
+      "default": null,
+      "title": "Source Name"
+    }
+  },
+  "required": [
+    "kind"
+  ],
+  "title": "FreshnessReason",
+  "type": "object"
+}

addedOutput schema / properties / answer_freshness

Added value: +{
+  "anyOf": [
+    {
+      "enum": [
+        "current",
+        "changed",
+        "unprovable"
+      ],
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "title": "Answer Freshness"
+}

addedOutput schema / properties / freshness_reasons

Added value: +{
+  "items": {
+    "$ref": "#/$defs/FreshnessReason"
+  },
+  "title": "Freshness Reasons",
+  "type": "array"
+}

First observedOct 28, 2026

TDQS

A3.9/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only carry the generic destructiveHint=true; the description adds the concrete consequence (question changes invalidate existing answers and grades) and the concurrency caveat that no revision CAS exists, which an agent must know to avoid silent clobbering. The recovery hint via test_case_read is a useful addition. It does not say whether expectation/advisory edits have similar invalidation effects, so not a full 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three telegraphic clauses, no filler, with the destructive consequence front-loaded. The clipped phrasing ("Edits supplied question/rubric") is efficient but slightly costs readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, non-atomic mutation with an output schema and full param coverage, the description covers impact, concurrency and recovery. The one remaining gap is whether changed conditions require a re-run via test_case_run, which an agent must infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, including the semantics of [] for expectations and advisory_expectations, so the schema carries the parameter burden. The description only loosely maps "question/rubric" onto those fields and adds no format or constraint detail beyond the schema; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource ("Edits supplied question/rubric") and names the sibling it interacts with (test_case_read), so an agent can distinguish it from test_case_create/remove. The term "rubric" is not used in the schema (which says expectations/advisory_expectations), a minor terminological mismatch that keeps it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It supplies operative context: which fields are editable, that question edits invalidate old answers and grades, and the recovery path via test_case_read after a lost response. It stops short of explicit when-not-to-use guidance or routing against test_case_create/test_case_remove.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Replai tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?