Design an eval suite
design_eval_suiteBuild an evaluation suite for a planned AI feature with AI PM Lab's method: test cases (typical, edge, out of scope, attacks), graders with a judge rubric, pass bars, and a JSON test set. Returns instructions to follow with the user's text.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "design_eval_suite"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| feature | No | What the AI feature does, who uses it, and what it must never do |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text | |
| links | Yes | The AI PM Lab pages the answer links to, where each idea can be tried hands-on |