run_batch_validation_tests
Validates multiple inputs against an llguidance grammar from a JSON test file, returning pass/fail statistics and details on failed tests.
Instructions
Run batch validation tests from a JSON file against an llguidance grammar. Test file should contain array of {input, should_parse, description?} objects. Returns high-level statistics (total, passed, failed, success_rate) and detailed information for failed tests.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"grammar": "<grammar>",
"test_file": "<test_file>"
},
"name": "run_batch_validation_tests"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| grammar | Yes | llguidance grammar string or path to grammar file (.lark, .grammar) | |
| test_file | Yes | Path to JSON test file with format: [{"input": "test", "should_parse": true}] or {"tests": [...]} |