Skip to main content
Glama
tylerscomic-lab

overfitting-audit-mcp

by tylerscomic-lab
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Hybrid

overfitting-audit-mcp

License: MIT Live on MCPize

An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.

The problem this solves

Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.

Related MCP server: backtest-reality-check

Tools

probability_of_backtest_overfitting

Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.

deflated_sharpe_ratio

Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.

minimum_backtest_length

The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.

purged_cv_split

Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.

Use it

Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.

Self-host:

npm install
node server.js

Part of the AlgoForge suite

Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.

License

MIT

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • backtester-mcp

    bcosm
    A
    license
    Not graded
    quality
    D
    maintenance
    Local-first backtesting engine with built-in overfitting detection (PBO, deflated Sharpe, bootstrap CI, walk-forward) and a native MCP server for AI agents to validate trading strategies.
    4
    Apache 2.0

  • backtest-reality-check

    maximdakulinin-boop
    A
    license
    Not graded
    quality
    B
    maintenance
    Checks whether a trading backtest survives its own statistics: deflated Sharpe, multiple-testing correction against a best-of-N-noise benchmark, minimum track record length, and fill realism. Takes no market data and no API keys, and cannot recommend a trade — it only reports that a result is weaker than claimed or not yet provable.
    MIT

  • crypto-quant-platform MCP

    FETKlOkAn2
    A
    license
    Not graded
    quality
    B
    maintenance
    Provides tools to research crypto trading strategies via backtesting, walk-forward validation, and paper trading, with a deflated-Sharpe overfitting check. Enables natural-language-driven analysis and interpretation of strategy performance.
    3
    Apache 2.0

  • regimen

    RaYYeR220
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables statistical validation of trading strategy equity curves via MCP, providing probabilistic and deflated Sharpe ratios, minimum track record length, regime-conditional performance attribution, and self-attack controls.
    49 npm
    MIT