QuantClaw Data

# ✅ Phase 41: Alert Backtesting - COMPLETED ## Summary Built comprehensive alert backtesting system to test alert strategies historically, measure signal quality, and identify false positive rates. Traders can now validate alert conditions before deploying them in production. ## Deliverables ### 1. Core Module (`modules/alert_backtest.py`) - 550 LOC - ✅ Historical data fetching with technical indicators (RSI, MACD, Bollinger Bands, SMAs, Volume) - ✅ Alert condition parsing and evaluation engine - ✅ Backtesting framework with forward returns (1d, 5d, 20d) - ✅ Hit rate calculation (did price move in expected direction?) - ✅ False positive/negative rate measurement - ✅ Profit factor calculation - ✅ Signal quality scoring (0-100) - ✅ Sharpe ratio and max drawdown metrics - ✅ Automatic direction detection (long/short bias) ### 2. CLI Commands ✅ **alert-backtest**: Test specific alert condition on historical data ```bash python cli.py alert-backtest AAPL --condition "rsi<30" --period 1y ``` ✅ **signal-quality**: Test and rank multiple conditions automatically ```bash python cli.py signal-quality TSLA --period 1y ``` ✅ **alert-potential**: Analyze trigger frequency and volatility ```bash python cli.py alert-potential NVDA --period 1y ``` ### 3. API Route (`src/app/api/v1/alert-backtest/route.ts`) - ✅ GET endpoint with query parameters - ✅ POST endpoint with JSON body - ✅ Three action modes: backtest, signal-quality, alert-potential - ✅ Error handling and timeout protection - ✅ JSON parsing from CLI output ### 4. Service Registry Updates - ✅ `src/app/services.ts` - Added 3 new services (alert_backtest, signal_quality, alert_potential) - ✅ `src/app/roadmap.ts` - Marked Phase 41 as "done" with 550 LOC ### 5. CLI Integration - ✅ `cli.py` - Registered alert_backtest module with command routing - ✅ Help text updated with new commands ## Test Results All tests passing ✅ ``` ✓ CLI commands work ✓ Multiple technical indicators supported (RSI, MACD, BB, Volume) ✓ Hit rate calculation working ✓ Signal quality scoring operational ✓ Alert potential analysis functional ``` ### Sample Test Outputs **AAPL - RSI < 30 (6 months)** - Total Signals: 16 - Hit Rate (1d): 50.00% - Hit Rate (20d): 81.25% - Signal Quality Score: 50.6/100 - Sharpe Ratio: -2.56 **TSLA - Signal Quality Analysis** - Top condition: RSI < 30 (Quality Score: 65.0/100) - Profit Factor: 20.24 - Hit Rate: 50.00% **NVDA - Alert Potential** - 126 trading days analyzed - RSI oversold triggers: 1 (0.8%) - Annualized volatility: 34.77% ## Technical Indicators Supported 1. **RSI**: Oversold/overbought conditions (rsi<30, rsi>70, etc.) 2. **MACD**: Bullish/bearish crossovers (macd>macd_signal, macd<0, etc.) 3. **Bollinger Bands**: Band touches (close<bb_lower, close>bb_upper) 4. **Moving Averages**: Trend conditions (close>sma_50, price<sma_200) 5. **Volume**: Volume spikes (volume_ratio>2, volume_ratio>3) ## Key Features ### Automatic Direction Detection System automatically determines expected price direction from condition: - RSI < 30 → Oversold → Expect bounce (long) - RSI > 70 → Overbought → Expect pullback (short) - MACD > 0 → Bullish → Long - Price > SMA → Uptrend → Long ### Signal Quality Score (0-100) Weighted composite metric: - 40% Hit Rate - 30% Profit Factor (capped at 3.0) - 30% Consistency (5d vs 1d hit rate stability) ### Comprehensive Metrics - Hit rates (1d, 5d, 20d horizons) - False positive rate - Average win/loss - Profit factor - Sharpe ratio - Max drawdown ## Data Source - **Provider**: Yahoo Finance (via yfinance) - **Cost**: $0 (free API) - **No API keys required** ## Files Created/Modified **Created:** - `/modules/alert_backtest.py` (550 lines) - `/src/app/api/v1/alert-backtest/route.ts` (200 lines) - `/modules/README_ALERT_BACKTEST.md` (documentation) - `/test_alert_backtest.sh` (test suite) **Modified:** - `/cli.py` (added alert_backtest module registration) - `/src/app/services.ts` (added 3 new services) - `/src/app/roadmap.ts` (marked Phase 41 as done) ## Integration Points ### With Phase 10 (Smart Alerts) - Can validate alert conditions before deployment - Suggested workflow: alert-potential → signal-quality → alert-backtest → deploy ### With Phase 40 (Smart Alert Delivery) - Validates conditions for multi-channel delivery - Quality score can inform alert priority levels ### Future Integration (Phase 80) - Dashboard visualization of backtest results - Historical performance charts - Comparison across multiple symbols ## Example Use Cases 1. **Validate Alert Before Deployment** ```bash # Test if RSI < 30 is a good alert for AAPL python cli.py alert-backtest AAPL --condition "rsi<30" --period 1y ``` 2. **Find Best Alert for a Symbol** ```bash # Test 10 common conditions and rank them python cli.py signal-quality TSLA --period 1y ``` 3. **Check Alert Frequency** ```bash # See how often different alerts would trigger python cli.py alert-potential NVDA --period 1y ``` ## Performance Notes - Typical execution time: 3-8 seconds per backtest - Supports periods: 1mo, 3mo, 6mo, 1y, 2y, 5y, max - Automatically handles missing data and forward return calculations - Filters out signals without sufficient forward data ## Limitations & Future Enhancements **Current Limitations:** - Single-condition alerts only (Phase 42 will add multi-condition DSL) - Long/short direction inferred from condition (not explicit parameter) - No slippage/commission modeling in profit factor **Planned Enhancements:** - Phase 42: Complex multi-condition rules (AND/OR logic) - Phase 80: Visual dashboard with charts - Integration with live alert monitoring - Portfolio-level backtest (multiple symbols) ## API Restart Note ⚠️ **Important**: The API endpoint has been created at `/api/v1/alert-backtest`, but requires a Next.js dev server restart to become active. The CLI commands work immediately. ```bash # To activate API endpoint (if needed): cd /home/quant/apps/quantclaw-data # Restart Next.js dev server (pm2 or similar) ``` ## Success Metrics ✅ **Phase Complete**: All requirements met - ✅ Module created with real implementation - ✅ CLI commands working (3 commands) - ✅ API route created - ✅ services.ts updated - ✅ roadmap.ts marked as "done" - ✅ Tests passing - ✅ Uses free APIs only (yfinance) - ✅ Real code, not stubs ## Documentation Complete documentation available in: - `modules/README_ALERT_BACKTEST.md` - User guide with examples - This file - Completion summary - Inline code comments and docstrings --- **Status**: ✅ COMPLETE **Lines of Code**: 550 **Data Sources**: Yahoo Finance (free) **API Keys Required**: None **Test Status**: All Passing **Deployment Status**: Ready (API needs server restart)