Skip to main content
Glama
Coval MCP Server

Coval MCP Server

Official
by coval-ai

Update test case

update_test_case
Destructive

Update a synthetic evaluation test case's scenario, expected behaviors, description, or script turns to keep test coverage accurate. Modify specific fields by test case ID without recreating the case.

Instructions

Update the scenario, expected behaviors, description, or script turns of one synthetic evaluation test case. To change turns, prefer the top-level script_turns field (server-side merge); simulation_metadata_input replaces the stored object wholesale.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "test_case_id": "<test_case_id>"
    },
    "name": "update_test_case"
  }
}

Input Schema

NameRequiredDescriptionDefault
input_strNoUpdated test input or scenario.
input_typeNoUpdated input type. Switching away from SCRIPT clears stored script_turns.
user_notesNoUpdated internal notes.
descriptionNoUpdated description.
metric_inputNoUpdated metric inputs.
script_turnsNoReplacement turn list, merged into stored simulation metadata server-side (sibling keys preserved). Preferred over simulation_metadata_input for turn edits. Entries: plain strings the persona speaks verbatim, or typed keypad presses {"type": "dtmf", "digits": "1"} (digits allow 0-9, *, # plus phone punctuation for full numbers). Never write DTMF as spoken text like "dtmf:1". When sent together with simulation_metadata_input, that object is replaced first and these turns are then merged into the result, so this field always wins for the script_turns key.
test_case_idYesThe unique ID of the test case to update.
expected_behaviorsNoUpdated list of expected agent behaviors.
simulation_metadata_inputNoUpdated simulation context. Replaces the stored object wholesale — include every key you want to keep. Prefer the top-level script_turns field for turn edits; if both are sent, this object is replaced first and script_turns is merged into the result. For scripted (input_type SCRIPT) cases you may set {"script_turns": [...]} here instead: entries are plain strings the persona speaks verbatim, or typed keypad presses {"type": "dtmf", "digits": "1"} (digits allow 0-9, *, # plus phone punctuation for full numbers). Never write DTMF as spoken text like "dtmf:1" — it would be read aloud. For SCENARIO cases against IVRs, instead instruct the persona in input_str to use its dtmf_tool for each keypress.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed6 schema fields changedv0.5.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / metric_input / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • changedInput schema / properties / script_turns / items / anyOf
      Previous value: -[
      -  {
      -    "minLength": 1,
      -    "type": "string"
      -  },
      -  {
      -    "anyOf": [
      -      {
      -        "additionalProperties": false,
      -        "properties": {
      -          "text": {
      -            "minLength": 1,
      -            "type": "string"
      -          },
      -          "type": {
      -            "const": "text",
      -            "type": "string"
      -          }
      -        },
      -        "required": [
      -          "type",
      -          "text"
      -        ],
      -        "type": "object"
      -      },
      -      {
      -        "additionalProperties": false,
      -        "properties": {
      -          "digits": {
      -            "allOf": [
      -              {
      -                "pattern": "^[0-9#*+(). -]+$"
      -              },
      -              {
      -                "pattern": "[0-9#*]"
      -              }
      -            ],
      -            "type": "string"
      -          },
      -          "type": {
      -            "const": "dtmf",
      -            "type": "string"
      -          }
      -        },
      -        "required": [
      -          "type",
      -          "digits"
      -        ],
      -        "type": "object"
      -      },
      -      {
      -        "additionalProperties": false,
      -        "properties": {
      -          "type": {
      -            "const": "skip",
      -            "type": "string"
      -          }
      -        },
      -        "required": [
      -          "type"
      -        ],
      -        "type": "object"
      -      }
      -    ]
      -  }
      -]New value: +[
      +  {
      +    "minLength": 1,
      +    "type": "string"
      +  },
      +  {
      +    "oneOf": [
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "text": {
      +            "minLength": 1,
      +            "type": "string"
      +          },
      +          "type": {
      +            "const": "text",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type",
      +          "text"
      +        ],
      +        "type": "object"
      +      },
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "digits": {
      +            "allOf": [
      +              {
      +                "pattern": "^[0-9#*+(). -]+$",
      +                "type": "string"
      +              },
      +              {
      +                "pattern": "[0-9#*]",
      +                "type": "string"
      +              }
      +            ],
      +            "type": "string"
      +          },
      +          "type": {
      +            "const": "dtmf",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type",
      +          "digits"
      +        ],
      +        "type": "object"
      +      },
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "type": {
      +            "const": "skip",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type"
      +        ],
      +        "type": "object"
      +      }
      +    ]
      +  }
      +]
    • changedInput schema / properties / simulation_metadata_input / additionalProperties
      Previous value: -trueNew value: +{}
    • removedInput schema / properties / simulation_metadata_input / properties / script_turns / items / $ref
      Removed value: -"#/properties/script_turns/items"
    • addedInput schema / properties / simulation_metadata_input / properties / script_turns / items / anyOf
      Added value: +[
      +  {
      +    "minLength": 1,
      +    "type": "string"
      +  },
      +  {
      +    "oneOf": [
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "text": {
      +            "minLength": 1,
      +            "type": "string"
      +          },
      +          "type": {
      +            "const": "text",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type",
      +          "text"
      +        ],
      +        "type": "object"
      +      },
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "digits": {
      +            "allOf": [
      +              {
      +                "pattern": "^[0-9#*+(). -]+$",
      +                "type": "string"
      +              },
      +              {
      +                "pattern": "[0-9#*]",
      +                "type": "string"
      +              }
      +            ],
      +            "type": "string"
      +          },
      +          "type": {
      +            "const": "dtmf",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type",
      +          "digits"
      +        ],
      +        "type": "object"
      +      },
      +      {
      +        "additionalProperties": false,
      +        "properties": {
      +          "type": {
      +            "const": "skip",
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "type"
      +        ],
      +        "type": "object"
      +      }
      +    ]
      +  }
      +]
  2. Changed7 schema fields changedv0.4.1
    • addedInput schema / properties / input_type
      Added value: +{
      +  "description": "Updated input type. Switching away from SCRIPT clears stored script_turns.",
      +  "enum": [
      +    "SCENARIO",
      +    "TRANSCRIPT",
      +    "IVR",
      +    "AUDIO",
      +    "MANUAL",
      +    "SCRIPT"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / script_turns
      Added value: +{
      +  "description": "Replacement turn list, merged into stored simulation metadata server-side (sibling keys preserved). Preferred over simulation_metadata_input for turn edits. Entries: plain strings the persona speaks verbatim, or typed keypad presses {\"type\": \"dtmf\", \"digits\": \"1\"} (digits allow 0-9, *, # plus phone punctuation for full numbers). Never write DTMF as spoken text like \"dtmf:1\". When sent together with simulation_metadata_input, that object is replaced first and these turns are then merged into the result, so this field always wins for the script_turns key.",
      +  "items": {
      +    "anyOf": [
      +      {
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      {
      +        "anyOf": [
      +          {
      +            "additionalProperties": false,
      +            "properties": {
      +              "text": {
      +                "minLength": 1,
      +                "type": "string"
      +              },
      +              "type": {
      +                "const": "text",
      +                "type": "string"
      +              }
      +            },
      +            "required": [
      +              "type",
      +              "text"
      +            ],
      +            "type": "object"
      +          },
      +          {
      +            "additionalProperties": false,
      +            "properties": {
      +              "digits": {
      +                "allOf": [
      +                  {
      +                    "pattern": "^[0-9#*+(). -]+$"
      +                  },
      +                  {
      +                    "pattern": "[0-9#*]"
      +                  }
      +                ],
      +                "type": "string"
      +              },
      +              "type": {
      +                "const": "dtmf",
      +                "type": "string"
      +              }
      +            },
      +            "required": [
      +              "type",
      +              "digits"
      +            ],
      +            "type": "object"
      +          },
      +          {
      +            "additionalProperties": false,
      +            "properties": {
      +              "type": {
      +                "const": "skip",
      +                "type": "string"
      +              }
      +            },
      +            "required": [
      +              "type"
      +            ],
      +            "type": "object"
      +          }
      +        ]
      +      }
      +    ]
      +  },
      +  "type": "array"
      +}
    • changedInput schema / properties / simulation_metadata_input / additionalProperties
      Previous value: -{}New value: +true
    • changedInput schema / properties / simulation_metadata_input / description
      Previous value: -"Updated simulation context."New value: +"Updated simulation context. Replaces the stored object wholesale — include every key you want to keep. Prefer the top-level script_turns field for turn edits; if both are sent, this object is replaced first and script_turns is merged into the result. For scripted (input_type SCRIPT) cases you may set {\"script_turns\": [...]} here instead: entries are plain strings the persona speaks verbatim, or typed keypad presses {\"type\": \"dtmf\", \"digits\": \"1\"} (digits allow 0-9, *, # plus phone punctuation for full numbers). Never write DTMF as spoken text like \"dtmf:1\" — it would be read aloud. For SCENARIO cases against IVRs, instead instruct the persona in input_str to use its dtmf_tool for each keypress."
    • addedInput schema / properties / simulation_metadata_input / properties
      Added value: +{
      +  "script_turns": {
      +    "items": {
      +      "$ref": "#/properties/script_turns/items"
      +    },
      +    "type": "array"
      +  }
      +}
    • addedInput schema / properties / test_case_id / maxLength
      Added value: +128
    • addedInput schema / properties / test_case_id / pattern
      Added value: +"^[A-Za-z0-9_-]+$"
  3. First observedv0.1.0

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=false; the description adds the crucial mechanism behind the destructiveness — simulation_metadata_input 'replaces the stored object wholesale' while script_turns merges and 'always wins'. That is real behavioral context not recoverable from annotations or schema. It doesn't mention permission requirements or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and followed by the precedence rule; no filler. Slightly dense and drops the article 'the' before 'simulation_metadata_input', but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter destructive update with no output schema, the description covers the highest-risk ambiguity (wholesale replacement vs merge) that an agent could get wrong. It leaves the return value unaddressed and doesn't warn about partial updates, keeping it just short of complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns the bump by documenting the merge-vs-replace precedence between script_turns and simulation_metadata_input, which the schema's per-field descriptions only partially capture. It doesn't elaborate on metric_input or user_notes beyond what the schema says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Update') and resource ('one synthetic evaluation test case') and enumerates the updatable fields (scenario, expected behaviors, description, script turns). It's clearly distinguishable from the read sibling get_test_case, though it doesn't explicitly contrast with create_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit routing rule between two overlapping fields: 'prefer the top-level script_turns field' over simulation_metadata_input for turn edits, which is genuine when-to-use guidance. It stops short of stating preconditions (e.g., whether the test case must exist or be in an editable state).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.