run_benchmark
Execute the benchmark suite to generate a dashboard with breakdowns by judge, category, and difficulty. Identify weak spots and evaluate system quality using precision, recall, F1, and false positive rates.
Instructions
Run the full benchmark suite and return a detailed dashboard with per-judge, per-category, and per-difficulty breakdowns. Includes precision, recall, F1, false positive rates, and individual case results. Use this to understand overall system quality and identify weak spots.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | Output format: markdown (full report), json (raw data), summary (key metrics only). Default: markdown | |
| judgeId | No | Optional: restrict benchmark to a single judge. One of: accessibility, agent-instructions, ai-code-safety, api-contract, api-design, authentication, backwards-compatibility, caching, ci-cd, cloud-readiness, code-structure, compliance, concurrency, configuration-management, cost-effectiveness, cybersecurity, data-security, data-sovereignty, database, dependency-health, documentation, error-handling, ethics-bias, framework-safety, hallucination-detection, iac-security, intent-alignment, internationalization, logging-privacy, logic-review, maintainability, model-fingerprint, multi-turn-coherence, observability, over-engineering, performance, portability, rate-limiting, reliability, scalability, security, software-practices, testing, ux, false-positive-review |