n8n Workflow Builder

# Release v1.10.0: Drift Detection System **Release Date:** 2025-12-16 ## 🎯 Overview This release introduces the **Drift Detection System** - a groundbreaking feature that detects workflow degradation over time by analyzing execution patterns. **The Problem:** Workflows age poorly. APIs change. Response formats drift. Timeouts creep in. Rate limits get added. Most tools just show "workflow is broken" - but not WHY or WHEN it broke. **The Solution:** Temporal drift analysis that compares baseline performance vs current state, identifies WHEN changes occurred, determines root causes with evidence, and suggests concrete fixes. ## ✨ New Features ### 1. Drift Detection **DriftDetector** automatically analyzes execution history: - Compares baseline (first 30%) vs current (last 30%) periods - Detects success rate drops (>15% threshold) - Identifies performance degradation (>50% duration change) - Spots new error patterns - Tracks error frequency increases (>2x) **Drift Patterns:** ```python { "success_rate_drift": "95% → 60% success (37% drop)", "performance_drift": "2.5s → 8.5s avg duration (3.4x slower)", "new_error_pattern": "Rate limit (429) errors appeared", "error_frequency_drift": "Auth errors increased 15x" } ``` ### 2. Pattern Analysis **DriftPatternAnalyzer** provides deep insights: - **Change Point Detection**: When did drift start? - **Gradual vs Sudden**: Was it overnight or slow degradation? - **Potential Causes**: List of likely root causes - **Recommendations**: What to investigate next ### 3. Root Cause Analysis **DriftRootCauseAnalyzer** determines WHY drift occurred: **Root Causes Identified:** - `api_rate_limit_introduced` - New rate limits added - `authentication_method_changed` - Auth format changed - `api_response_format_changed` - JSON structure changed - `credential_expiration` - API keys expired - `external_service_slowdown` - Upstream API degraded - `workflow_degradation` - Multiple factors **Evidence-Based:** ```python { "root_cause": "api_rate_limit_introduced", "confidence": 0.85, "evidence": [ "Rate limit (429) errors appeared where none existed before", "Errors likely started after API provider update" ], "recommended_action": "Add request throttling or implement exponential backoff" } ``` ### 4. Fix Suggestions **DriftFixSuggester** generates actionable fixes: **For Rate Limits:** - Add delay between requests (1-2s) - Implement exponential backoff - Batch requests to reduce volume **For Auth Changes:** - Refresh API credentials - Check auth header format (Bearer → Token) - Review API documentation **For Response Format Changes:** - Update JSON field paths - Add fallback for backward compatibility - Add data validation **For Performance Issues:** - Increase timeout values - Add caching layer - Optimize queries and reduce payload size ## 🛠️ New MCP Tools ### 1. `detect_workflow_drift` Detects workflow degradation by comparing execution patterns over time. **Parameters:** - `workflow_id` (required): Workflow to analyze - `lookback_days` (optional): Days of history (default: 30) **Returns:** - Drift detected (yes/no) - Severity (critical/warning/info) - Metrics comparison (baseline vs current) - Detected patterns - Next steps **Example:** ```python detect_workflow_drift(workflow_id="abc123", lookback_days=30) # Returns: # 🔴 Drift Detected - Severity: CRITICAL # # Metrics Comparison: # Success Rate: # - Baseline: 95.0% # - Current: 60.0% # - Change: -35.0% # # Avg Duration: # - Baseline: 2500ms # - Current: 8500ms # # Detected Patterns: # 🔴 success_rate_drift # - Success rate decreased by 35.0% # # ⚠️ new_error_pattern # - New error types appeared: rate_limit ``` ### 2. `analyze_drift_pattern` Deep analysis of a specific drift pattern. **Parameters:** - `workflow_id` (required) - `pattern_type` (required): success_rate_drift, performance_drift, new_error_pattern, error_frequency_drift **Returns:** - When drift started - Gradual vs sudden change - Potential causes - Recommendations **Example:** ```python analyze_drift_pattern( workflow_id="abc123", pattern_type="new_error_pattern" ) # Returns: # Pattern Analysis: new_error_pattern # Severity: WARNING # Started Around: 2025-12-10T00:00:00Z # Change Type: Sudden # # Potential Causes: # - API provider added rate limiting # - New permission requirements added # # Recommendation: # Review API provider changelog and test failing requests manually ``` ### 3. `get_drift_root_cause` Evidence-based root cause determination. **Parameters:** - `workflow_id` (required) - `lookback_days` (optional): Default 30 **Returns:** - Root cause identification - Confidence score - Evidence list - Recommended action **Example:** ```python get_drift_root_cause(workflow_id="abc123") # Returns: # Root Cause Analysis # Root Cause: api_rate_limit_introduced # Confidence: 85% # # Evidence: # - Rate limit errors increased 15.0x # - API provider likely reduced rate limits # # Recommended Action: # Reduce request frequency or implement request queuing ``` ### 4. `get_drift_fix_suggestions` Actionable fix recommendations with confidence scores. **Parameters:** - `workflow_id` (required) **Returns:** - Root cause - Confidence score - Specific fix suggestions per node - Testing recommendations **Example:** ```python get_drift_fix_suggestions(workflow_id="abc123") # Returns: # Fix Suggestions # Root Cause: api_rate_limit_introduced # Confidence: 85% # # Suggested Fixes: # 1. Add delay between requests # Node: HTTP Request # Suggestion: Add a 'Wait' node before this HTTP request with 1-2 second delay # Confidence: 85% # # 2. Implement exponential backoff # Node: HTTP Request # Suggestion: Enable 'Retry On Fail' with exponential backoff in node settings # Confidence: 90% # # Testing Recommendations: # - Test fixes in development environment first # - Monitor execution success rate after applying fixes # - Compare error patterns before and after changes ``` ## 📊 How It Works ### Baseline vs Current Comparison ``` Execution History (100 executions) ├── Baseline Period (first 30%): Executions 1-30 │ └── Metrics: 95% success, 2.5s avg, 1 auth error └── Current Period (last 30%): Executions 70-100 └── Metrics: 60% success, 8.5s avg, 15 auth errors Analysis: → Success rate drift: -35% (CRITICAL) → Performance drift: +3.4x duration (WARNING) → Error frequency drift: 15x increase in auth errors (WARNING) ``` ### Change Point Detection Finds WHEN drift started: ``` Week 1-3: 95% success ✅ Week 4: 60% success ❌ ← Change detected here ``` **Methods:** - Rolling window analysis - Statistical change detection - Gradual vs sudden classification ### Error Pattern Recognition Tracks error evolution: ``` Before: [timeout: 1] After: [timeout: 1, rate_limit: 8, auth: 15] → New error patterns emerged → Auth errors spiked 15x ``` ## 🎯 Use Cases ### Use Case 1: API Breaking Change Detection **Scenario:** Workflow worked for 2 months, suddenly all executions fail ```python # 1. Detect drift drift = detect_workflow_drift(workflow_id="api-sync") # → 🔴 Critical: Success rate dropped from 95% to 20% # → Pattern: New error type "authentication" appeared # 2. Analyze pattern pattern = analyze_drift_pattern( workflow_id="api-sync", pattern_type="new_error_pattern" ) # → Started: 2025-12-10 00:00 UTC (overnight) # → Type: Sudden change # → Cause: Authentication method likely changed # 3. Get root cause cause = get_drift_root_cause(workflow_id="api-sync") # → Root Cause: authentication_method_changed # → Confidence: 80% # → Evidence: All failures started on specific date # → Action: Check if auth format changed (Bearer → Token) # 4. Get fix suggestions fixes = get_drift_fix_suggestions(workflow_id="api-sync") # → Update authentication header format # → Refresh API credentials # → Test manually with Postman ``` ### Use Case 2: Rate Limit Introduction **Scenario:** Workflow runs fine, then starts failing intermittently ```python drift = detect_workflow_drift(workflow_id="data-fetch") # → ⚠️ Warning: New error pattern "rate_limit" # → 8 rate limit (429) errors in last period cause = get_drift_root_cause(workflow_id="data-fetch") # → Root Cause: api_rate_limit_introduced # → Confidence: 85% # → Evidence: 429 errors appeared where none existed # → Action: Add request throttling fixes = get_drift_fix_suggestions(workflow_id="data-fetch") # → Add 1-2s delay between requests # → Implement exponential backoff # → Consider request batching ``` ### Use Case 3: Performance Degradation **Scenario:** Workflow getting slower over time ```python drift = detect_workflow_drift(workflow_id="etl") # → ⚠️ Warning: Performance drift # → Duration increased from 2.5s to 8.5s (3.4x) pattern = analyze_drift_pattern( workflow_id="etl", pattern_type="performance_drift" ) # → Started: ~2 weeks ago # → Type: Gradual degradation # → Causes: API response times, database performance fixes = get_drift_fix_suggestions(workflow_id="etl") # → Increase timeout values # → Add caching for frequently accessed data # → Optimize queries and reduce payload size ``` ### Use Case 4: Response Format Drift **Scenario:** Workflow succeeds but data processing breaks ```python drift = detect_workflow_drift(workflow_id="processor") # → ⚠️ Warning: New error "json_parse" # → Parsing errors increased cause = get_drift_root_cause(workflow_id="processor") # → Root Cause: api_response_format_changed # → Confidence: 75% # → Evidence: JSON parsing errors started occurring # → Action: Compare old/new API responses fixes = get_drift_fix_suggestions(workflow_id="processor") # → Update field mappings: data.user.email → data.profile.email # → Add fallback: {{$json.profile?.email || $json.user?.email}} # → Add validation to detect format changes early ``` ## 🔬 Technical Details ### Drift Detection Algorithm ```python # 1. Collect execution history (last 100 executions) # 2. Split into baseline (first 30%) and current (last 30%) # 3. Calculate metrics for each period: # - Success rate # - Average duration # - Error types and frequencies # 4. Compare metrics: # - Success rate change > 15% → drift # - Duration change > 50% → drift # - New error types → drift # - Error frequency > 2x → drift # 5. Classify severity: # - Critical: >30% success drop or >2x duration # - Warning: >15% success drop or >1.5x duration # - Info: Minor changes detected ``` ### Pattern Analysis **Change Point Detection:** - Rolling window average (window size: 10% of history) - Detects largest metric jump - Classifies as gradual or sudden **Root Cause Confidence:** - Based on pattern matching - Evidence strength - Timing correlation - 0.0 = unknown, 1.0 = certain ### Fix Suggestion Generation **Confidence Scores:** - High (>0.85): Strong evidence, clear fix - Medium (0.70-0.85): Likely cause, recommended fix - Low (<0.70): Possible cause, general fix ## 💡 Unique Value Proposition ### What Other Tools Do: - ❌ Show current error: "Request failed with 401" - ❌ Say: "Workflow is broken" - ❌ No historical context ### What Drift Detection Does: - ✅ Shows WHY: "Auth method changed from Bearer to Token" - ✅ Shows WHEN: "Change occurred on 2025-12-10" - ✅ Shows WHAT: "Update header format in HTTP Request node" - ✅ Shows HOW: "Change 'Bearer {token}' to 'Token {token}'" - ✅ Confidence: "85% confident this is the cause" ### Example Comparison: **Other Tool:** ``` Error: Field 'email' not found in response ``` **Drift Detection:** ``` 🔴 Drift Detected: API response format changed Evidence: - Parsing errors started on 2025-12-10 - Field path changed from 'user.email' to 'profile.email' - Same pattern observed in 3 other workflows Root Cause: api_response_format_changed (75% confidence) Fixes: 1. Update expression in "Transform Data" node From: {{$json.user.email}} To: {{$json.profile.email}} 2. Add fallback for backward compatibility Use: {{$json.profile?.email || $json.user?.email}} 3. Add validation to detect future changes Add IF node to check field exists ``` ## 📈 Impact ### Before Drift Detection: ``` Week 1-3: Workflow runs fine ✅ Week 4: Everything breaks ❌ Engineer: "What happened? When did this break? What changed?" *Spends hours debugging* *Checks API docs* *Tests manually* *Finally finds: Auth method changed* ``` ### After Drift Detection: ``` Week 4: detect_workflow_drift() System: "🔴 Auth drift detected on 2025-12-10" System: "Auth method changed from Bearer to Token" System: "Fix: Update HTTP Request node auth header" Engineer: *Applies fix in 2 minutes* ``` ## 🎉 Benefits ✅ **Proactive Monitoring** - Detect issues before complete failure ✅ **Root Cause Analysis** - Know WHY, not just WHAT broke ✅ **Temporal Context** - Know WHEN changes occurred ✅ **Evidence-Based** - Decisions backed by data ✅ **Actionable Fixes** - Concrete steps, not vague advice ✅ **Confidence Scores** - Know how certain the analysis is ✅ **Time Savings** - Minutes instead of hours debugging ## 🔧 Files Changed ### New Files - `src/n8n_workflow_builder/drift/__init__.py` - `src/n8n_workflow_builder/drift/detector.py` (800+ lines) - DriftDetector (200 lines) - DriftPatternAnalyzer (250 lines) - DriftRootCauseAnalyzer (200 lines) - DriftFixSuggester (150 lines) ### Modified Files - `src/n8n_workflow_builder/server.py` (+200 lines) - Added 4 drift detection tool definitions - Added 4 drift detection tool handlers - Integrated drift analysis components ## 🐛 Bug Fixes - Fixed method name: `list_executions` → `get_executions` (correct N8nClient method) ## 📦 Dependencies No new dependencies. Uses existing execution history from n8n API. ## 🚀 Getting Started ```python # 1. Detect if workflow has degraded drift = detect_workflow_drift(workflow_id="abc123") # 2. If drift detected, analyze specific patterns pattern = analyze_drift_pattern( workflow_id="abc123", pattern_type="success_rate_drift" ) # 3. Get root cause with evidence cause = get_drift_root_cause(workflow_id="abc123") # 4. Get actionable fix suggestions fixes = get_drift_fix_suggestions(workflow_id="abc123") # 5. Apply fixes and monitor ``` ## 🎯 Future Enhancements Potential improvements: - Predictive drift detection (warn before failure) - Machine learning on fix effectiveness - Cross-workflow pattern correlation - Automatic fix application - Drift alerts via webhook/email - Historical drift reports - Drift trend visualization --- **Full Changelog**: v1.9.2...v1.10.0