Money Mind — the judge

The tools cover distinct statistical tests and checks, with clear guidance in descriptions for when to use each (e.g., judge-nano vs judge-lite vs judge-batch). However, there is some overlap between multipletest and judge-nano for single-series selection correction, and the inclusion of web-fetching tools (v1_extract, factcheck) alongside backtest validation may momentarily confuse an agent about scope.

Tool names mix conventions: some are squished compound words (clusteredt, deflatedsharpe, multipletest, abtest) while others use underscores (check_eval_selection, check_price_data) or hyphens (judge-batch, judge-lite). This inconsistency makes the set feel less predictable.

17 tools is on the heavy side for a judging service, with several tools (judge-nano, judge-lite, multipletest, deflatedsharpe) addressing overlapping statistical corrections at different price points. While each has a niche, the volume may overwhelm an agent.

The surface covers a wide range of validation gates: multiple-testing correction, clustering, drawdown, regime shifts, sample size, data quality, and comprehensive audits. Minor gaps exist (e.g., no explicit cross-validation or walk-forward tool), but the core judging needs are well met.