Skip to main content
Glama

CallMCP — Phone Call MCP Server

Run Agent Eval

run_eval
Destructive

Run a saved test scenario against one of the business's agents and grade the agent's replies: pass eval_id for one scenario, or agent_id for every scenario saved for that agent (see list_evals). The test is a simulated text conversation with the agent's current live configuration; no phone call is placed, no customer is contacted and the agent's configuration is not changed. Each run uses paid voice-agent provider capacity and is recorded with its result. Returns pass or fail per scenario with the grader's reasons.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "run_eval"
  }
}

Input Schema

NameRequiredDescriptionDefault
waitNoWait for the grade before returning (default true).
eval_idNoRun this one saved scenario (from list_evals). Either this or agent_id is required.
agent_idNoRun every saved scenario for this agent. Either this or eval_id is required.
max_wait_msNoPer-run wait cap in ms when wait is true (default 60000, max 110000).

Output Schema

NameRequiredDescriptionDefault
errorNoPresent when success is false
totalNo
failedNo
passedNo
run_idNo
statusNo
resultsNo
successYesWhether the tool completed successfully
completedNo
provider_run_idNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Against annotations that only mark non-read-only/destructive/open-world, the description discloses the things an agent actually needs: it is a simulated text conversation against live config, no phone call or customer is touched, config is unchanged, runs consume paid voice-provider capacity, and each run is persisted with its result. That is a genuinely useful disclosure of cost and side effects that annotations alone cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Sentences are front-loaded and dense: capability first, addressing modes second, safety/cost caveats third, return value last. The closing sentence restating the pass/fail return is partly redundant given an output schema exists, which is the only wasted line.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter, output-schema-backed execution tool, the description covers what it does, how to scope it, what it does not affect, what it costs, and what comes back. An agent could call it correctly without opening any other artifact except list_evals for ids.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes slightly beyond the schema by framing eval_id and agent_id as two addressing modes for the same operation and referencing list_evals as the id source. It still says nothing about wait or max_wait_ms, relying entirely on the schema there.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb pair ("run ... grade") against a precise resource ("a saved test scenario" against "one of the business's agents") and immediately states the exact outcome (pass or fail per scenario with grader reasons). It is clearly separable from list_evals and create_eval, which manage scenarios rather than execute them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit selection logic: eval_id runs one scenario, agent_id runs every scenario saved for that agent, and it points at list_evals as the source of ids. There is no explicit "do not use this when..." exclusion, and the motivation for choosing a single scenario versus a whole agent sweep is left implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.