Compare reports from before and after a change
compare_reportsCompare before and after test reports of the same check to get a verdict (IMPROVED, REGRESSED, MIXED, UNCHANGED), see what happened to each problem, and get a Markdown block for your pull request.
Instructions
Compares reports of the same check from before and after a change, such as main and a branch, and returns a verdict (IMPROVED, PARTLY_IMPROVED, REGRESSED, MIXED, UNCHANGED, INCONCLUSIVE or NOT_COMPARABLE), what happened to each problem, and a Markdown block for a pull request. A problem counts as gone only when a complete run after the change checked it, so a run that stopped early is never a fix. Give several reports per side for repeated runs.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"before": "<before>",
"after": "<after>"
},
"name": "compare_reports"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| after | Yes | Reports of the same check from after the change. | |
| before | Yes | Reports from before the change: report.json files or their folders. | |
| afterLabel | No | Name for the after side, such as the branch. | |
| beforeLabel | No | Name for the before side, such as main. |