agent_eval_plan
Build an AI agent evaluation plan from defined capabilities and risks: size a golden set, pick metrics, write a judge rubric, add red-team cases, and set regression cadence.
Instructions
Design an evaluation plan for an AI agent: golden set sizing, metrics, judge rubric, red-team cases and regression cadence from explicit capabilities and risks.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"agent": "<agent>",
"capabilities": "<capabilities>",
"risks": "<risks>"
},
"name": "agent_eval_plan"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| agent | Yes | ||
| risks | Yes | ||
| tools | No | ||
| capabilities | Yes | ||
| releaseCadence | No | weekly |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | ||
| agent | No | ||
| metrics | No | ||
| redTeam | No | ||
| sources | No | ||
| goldenSet | No | ||
| nextTools | No | ||
| validated | No | ||
| limitation | No | ||
| judgeRubric | No | ||
| requirements | No | ||
| mutatingTools | No |