Skip to main content
Glama
Sign In
overfitting-audit-mcp

overfitting-audit-mcp

by tylerscomic-lab
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Hybrid

overfitting-audit-mcp

License: MIT Live on MCPize

An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.

The problem this solves

Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.

Related MCP server: backtest-reality-check

Tools

probability_of_backtest_overfitting

Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.

deflated_sharpe_ratio

Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.

minimum_backtest_length

The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.

purged_cv_split

Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.

Use it

Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.

Self-host:

npm install
node server.js

Part of the AlgoForge suite

Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.

License

MIT

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    This server cannot be deployed

    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Resources

    Related MCP Connectors

    Related MCP Servers

    • backtester-mcp

      bcosm
      A
      license
      Not graded
      quality
      D
      maintenance
      Local-first backtesting engine with built-in overfitting detection (PBO, deflated Sharpe, bootstrap CI, walk-forward) and a native MCP server for AI agents to validate trading strategies.
      4
      Apache 2.0

    • backtest-reality-check

      maximdakulinin-boop
      A
      license
      Not graded
      quality
      D
      maintenance
      Checks whether a trading backtest survives its own statistics: deflated Sharpe, multiple-testing correction against a best-of-N-noise benchmark, minimum track record length, and fill realism. Takes no market data and no API keys, and cannot recommend a trade — it only reports that a result is weaker than claimed or not yet provable.
      MIT

    • crypto-quant-platform MCP

      FETKlOkAn2
      A
      license
      Not graded
      quality
      C
      maintenance
      Provides tools to research crypto trading strategies via backtesting, walk-forward validation, and paper trading, with a deflated-Sharpe overfitting check. Enables natural-language-driven analysis and interpretation of strategy performance.
      2
      Apache 2.0

    • regimen

      RaYYeR220
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables statistical validation of trading strategy equity curves via MCP, providing probabilistic and deflated Sharpe ratios, minimum track record length, regime-conditional performance attribution, and self-attack controls.
      71 npm
      MIT