Skip to main content
Glama
Coval MCP Server

Coval MCP Server

Official
by coval-ai

Create run

create_run
Destructive

Launch an evaluation run for a specific agent, persona, and test set, with optional tags to filter results and configuration for iterations and concurrency.

Instructions

Launch an evaluation for an agent, persona, and test set. Optional tags support result filtering.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "agent_id": "<agent_id>",
      "persona_id": "<persona_id>",
      "test_set_id": "<test_set_id>"
    },
    "name": "create_run"
  }
}

Input Schema

NameRequiredDescriptionDefault
tagsNoTags for categorizing and filtering the run (max 20 tags, 200 chars each).
optionsNoRun configuration options
agent_idYesThe unique ID of the agent to evaluate.
metadataNoCustom metadata for tracking purposes
metric_idsNoOptional list of metric IDs to evaluate. Uses agent defaults if omitted.
persona_idYesThe unique ID of the persona to use.
test_set_idYesThe unique ID of the test set to run against.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changedv0.5.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / metadata / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • removedInput schema / properties / options / additionalProperties
      Removed value: -false
  2. Changed6 schema fields changedv0.4.1
    • changedInput schema / properties / agent_id / description
      Previous value: -"The unique ID of the agent to evaluate. Get this from list_agents."New value: +"The unique ID of the agent to evaluate."
    • removedInput schema / properties / options / properties / concurrency / description
      Removed value: -"Number of parallel simulations (1-5, default 1)"
    • removedInput schema / properties / options / properties / iteration_count / description
      Removed value: -"Number of iterations per test case (1-10, default 1)"
    • changedInput schema / properties / persona_id / description
      Previous value: -"The unique ID of the persona to use. Get this from list_personas."New value: +"The unique ID of the persona to use."
    • changedInput schema / properties / tags / description
      Previous value: -"Tags for categorizing and filtering the run (max 20 tags, 200 chars each). Filter later with list_runs filter='tag=\"regression\"'."New value: +"Tags for categorizing and filtering the run (max 20 tags, 200 chars each)."
    • changedInput schema / properties / test_set_id / description
      Previous value: -"The unique ID of the test set to run against. Get this from list_test_sets."New value: +"The unique ID of the test set to run against."
  3. First observedv0.1.0

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, openWorldHint=true, and non-idempotent, so the agent knows this is a consequential write. The description adds nothing beyond that: it doesn't say the run is launched asynchronously, what happens to prior results, whether execution consumes quota, or that re-invoking creates a duplicate run rather than retrying.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core action before the optional-tags detail. Nothing is padded, though it is arguably too terse for a 7-parameter destructive tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The schema is fully described and carries the required IDs and nested options, so the description is minimally adequate. But for a non-idempotent, destructive, open-world launch tool with no output schema, the absence of any note on what is created/returned or execution implications leaves a real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description only echoes the tags filtering note already present in the schema and says nothing about the nested options (concurrency, iteration_count) or metric_ids defaulting behavior, so no value is added.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: 'Launch an evaluation' for an agent, persona, and test set, which maps cleanly to create_run and distinguishes it from get_run/list_runs. It doesn't explicitly name the read-only siblings, but the intent is unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this versus alternatives (e.g., create_scheduled_run for repeated runs, or list_run_templates). The only usage-adjacent content is 'Optional tags support result filtering', which is a feature note rather than selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.