analyze_overfit_probability
analyze_overfit_probabilityEstimate the Probability of Backtest Overfitting across a set of candidate configurations via combinatorially-symmetric cross-validation, for a required experiment protocolId. Use after testing several parameterizations to check whether keeping the in-sample best generalizes out-of-sample.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| blocks | No | Even number of contiguous time blocks to split the window into for cross-validation; defaults to 12. | |
| metric | No | Per-block performance metric CSCV ranks configurations by; defaults to 'sharpe'. | |
| source | Yes | ||
| symbol | Yes | ||
| configs | Yes | Two or more candidate configurations (input-override sets) to compare for selection overfitting. Each is run over the same window. | |
| endDate | Yes | ||
| startDate | Yes | ||
| timeframe | Yes | ||
| protocolId | Yes | Experiment protocol id returned by define_experiment_protocol for this exact overfitting check. | |
| caseStudyId | No | Optional: bind this call to one of your case studies. Defaults to the bound run's case study. | |
| researchRunId | No | Optional: bind this call to one of your existing research runs. Defaults to this session's most recent unfinished run. | |
| mcpIdempotencyKey | No | Unique key for this logical action. Reuse it only when retrying the same arguments. Required for hosted trust-creating actions when no idempotency header is supplied. |