Run Agent Eval
run_evalRun a saved test scenario against one of the business's agents and grade the agent's replies: pass eval_id for one scenario, or agent_id for every scenario saved for that agent (see list_evals). The test is a simulated text conversation with the agent's current live configuration; no phone call is placed, no customer is contacted and the agent's configuration is not changed. Each run uses paid voice-agent provider capacity and is recorded with its result. Returns pass or fail per scenario with the grader's reasons.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "run_eval"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| wait | No | Wait for the grade before returning (default true). | |
| eval_id | No | Run this one saved scenario (from list_evals). Either this or agent_id is required. | |
| agent_id | No | Run every saved scenario for this agent. Either this or eval_id is required. | |
| max_wait_ms | No | Per-run wait cap in ms when wait is true (default 60000, max 110000). |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| error | No | Present when success is false | |
| total | No | ||
| failed | No | ||
| passed | No | ||
| run_id | No | ||
| status | No | ||
| results | No | ||
| success | Yes | Whether the tool completed successfully | |
| completed | No | ||
| provider_run_id | No |