Eval report
eval_reportCompares board vs chat workflow trials using label conventions, returning per-arm medians, totals, matched pairs with token ratios, and a one-line summary.
Instructions
Compare board-workflow vs chat-workflow trials using label conventions: experiment:board / experiment:chat marks a ticket's arm, and an optional pair: label ties a board trial to its chat counterpart. Returns every labeled trial (tokens and $ cost from the work log, additions/deletions, wall-clock days, rework = linked bugs within 7 days of completion), per-arm medians/totals (including totalCost), matched pairs with a token ratio, and a one-line summary.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"project": "<project>"
},
"name": "eval_report"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes |