# Session-Buddy V3 to V4 Migration Guide
**Date:** 2026-02-10
**Status:** ✅ Production Ready
**Breaking Changes:** None (V4 is purely additive)
---
## Executive Summary
V4 introduces enterprise-grade analytics, real-time monitoring, cross-session learning, and multi-modal skill support. **This migration is completely backward compatible** - all V3 functionality remains intact.
**Key Benefits:**
- 📊 **Real-Time Dashboards** - WebSocket streaming of live metrics
- 🤖 **Predictive Analytics** - ML-based skill success prediction
- 🧪 **A/B Testing** - Experiment with recommendation strategies
- 👥 **Collaborative Filtering** - Learn from similar users
- 🔗 **Tool Integration** - Crackerjack, IDE, CI/CD hooks
- 📚 **Skills Taxonomy** - Organized categories and dependencies
**Migration Impact:**
- ✅ **Zero Downtime** - V4 tables added independently
- ✅ **No Breaking Changes** - All V3 features work unchanged
- ✅ **Rollback Safe** - Complete V4__down.sql available
- ✅ **Data Preservation** - No existing data modified
---
## Migration Overview
### What Changes
| Component | V3 | V4 |
|-----------|----|----|
| **Schema** | 3 tables + views | **14 new tables** + 6 new views |
| **Real-Time** | Manual queries | **WebSocket server** + metrics cache |
| **Analytics** | Basic aggregations | **Predictive models** + A/B testing |
| **Learning** | Single-session | **Cross-user collaborative filtering** |
| **Integration** | None | **Crackerjack/IDE/CI/CD** |
| **Taxonomy** | None | **Skills categories + modalities** |
### What Stays The Same
✅ **All V3 tables remain unchanged**
✅ **All V3 MCP tools continue working**
✅ **Existing queries and views functional**
✅ **No API changes to V3 functionality**
---
## Pre-Migration Checklist
### 1. Backup Database
**CRITICAL:** Always backup before migration:
```bash
# Locate your database
# Default: ~/.claude/data/session_buddy.db
# Create backup
cp ~/.claude/data/session_buddy.db ~/.claude/data/session_buddy.db.v3.backup
# Verify backup
ls -lh ~/.claude/data/session_buddy.db.v3.backup
```
### 2. Check Current Version
```python
from session_buddy.storage.migrations.base import MigrationManager
manager = MigrationManager(db_path="~/.claude/data/session_buddy.db")
print(f"Current version: {manager.get_current_version()}") # Should print 3
```
### 3. Verify Dependencies
```bash
cd /path/to/session-buddy
# Check V4 dependencies are installed
uv pip check
# Or install if needed
uv sync --all-extras
```
**V4 New Dependencies:**
- `websockets>=15.0` - WebSocket server
- `scikit-learn>=1.6.0` - Predictive models
- `scipy>=1.15.0` - Statistical analysis
---
## Migration Steps
### Step 1: Apply V4 Migration
**Automatic Migration (Recommended):**
```python
from pathlib import Path
from session_buddy.storage.migrations.base import MigrationManager
# Automatic migration
db_path = Path.home() / ".claude" / "data" / "session_buddy.db"
manager = MigrationManager(db_path=db_path)
manager.migrate()
# Output:
# Applying migration V4__phase4_extensions__up.sql
# ✓ Created 11 new tables
# ✓ Created 6 new views
# ✓ Created 3 triggers
# ✓ Migration V4 complete
```
**Manual Migration:**
```bash
# Apply migration SQL directly
sqlite3 ~/.claude/data/session_buddy.db < session_buddy/storage/migrations/V4__phase4_extensions__up.sql
```
### Step 2: Verify Migration
```sql
-- Check V4 tables exist
SELECT name FROM sqlite_master
WHERE type='table'
AND name IN (
'skill_metrics_cache',
'skill_time_series',
'skill_anomalies',
'skill_community_baselines',
'skill_user_interactions',
'ab_test_configs',
'skill_categories',
'skill_dependencies'
);
-- Expected: 8 rows (one per table)
-- Check V4 views exist
SELECT name FROM sqlite_master
WHERE type='view'
AND name LIKE 'v_%';
-- Expected: 6+ views (existing + new)
```
### Step 3: Initialize Taxonomy (Optional but Recommended)
```bash
# Run taxonomy initialization script
cd /path/to/session-buddy
python scripts/initialize_taxonomy.py
# Output:
# Initializing skills taxonomy...
# ✓ Initialized 6 categories
# ✓ Initialized 4 modality types
# ✓ Initialized 4 dependencies
# Taxonomy initialization complete!
```
This pre-populates:
- **6 Categories**: Code Quality, Testing, Documentation, Build & Deploy, Git & Version Control, Linting & Formatting
- **4 Modalities**: ruff-check (code→diagnostics), pytest-run (testing→test_results), sphinx-build (documentation→html_docs), docker-build (deployment→docker_image)
- **4 Dependencies**: ruff-check ↔ black-format, pytest-run ↔ coverage-report, git-commit → git-push, docker-build → k8s-deploy
---
## Post-Migration Setup
### 1. Start WebSocket Server (Optional)
For real-time dashboards:
```python
from session_buddy.realtime import RealTimeMetricsServer
server = RealTimeMetricsServer(
host="localhost",
port=8765,
db_path="~/.claude/data/session_buddy.db"
)
await server.start()
# Broadcasting at ws://localhost:8765 every 1 second
```
Or standalone:
```bash
python examples/run_websocket_server.py
```
### 2. Start Prometheus Exporter (Optional)
For metrics scraping:
```python
from session_buddy.realtime import PrometheusExporter
exporter = PrometheusExporter(port=9090)
exporter.start()
# Metrics at http://localhost:9090/metrics
```
### 3. Test New MCP Tools
Verify Phase 4 tools are registered:
```python
# Via MCP client
result = await call_tool("get_real_time_metrics", {"limit": 5})
print(result["top_skills"])
result = await call_tool("detect_anomalies", {"threshold": 2.0})
print(result["anomalies"])
result = await call_tool("get_collaborative_recommendations",
{"user_id": "user123", "limit": 5})
print(result["recommendations"])
```
---
## Data Backfill (Optional)
If migrating existing data:
### 1. Backfill Time-Series Data
```python
from datetime import datetime, timedelta
from session_buddy.storage.skills_storage import SkillsStorage
storage = SkillsStorage(db_path="~/.claude/data/session_buddy.db")
# Backfill last 30 days
end_date = datetime.now()
start_date = end_date - timedelta(days=30)
# Aggregate historical invocations into hourly time-series
for hour in range(30 * 24): # 30 days * 24 hours
hour_timestamp = start_date + timedelta(hours=hour)
# Insert aggregated hourly data
with storage._get_connection() as conn:
conn.execute(
"""
INSERT INTO skill_time_series (
skill_name, timestamp, invocation_count,
completion_rate, avg_duration_seconds, unique_sessions
)
SELECT
skill_name,
? as timestamp,
COUNT(*) as invocation_count,
AVG(CASE WHEN completed = 1 THEN 1.0 ELSE 0.0 END) as completion_rate,
AVG(duration_seconds) as avg_duration_seconds,
COUNT(DISTINCT session_id) as unique_sessions
FROM skill_invocation
WHERE datetime(invoked_at) >= datetime(?)
AND datetime(invoked_at) < datetime(?, '+1 hour')
GROUP BY skill_name
""",
(hour_timestamp.isoformat(), hour_timestamp.isoformat(), hour_timestamp.isoformat())
)
conn.commit()
```
### 2. Calculate Skill Dependencies
```python
# Calculate co-occurrence patterns
result = storage.update_skill_dependencies(min_co_occurrence=5)
print(f"Updated {result['dependencies_created']} dependencies")
```
### 3. Initialize Community Baselines
```python
from session_buddy.analytics import get_collaborative_engine
engine = get_collaborative_engine()
engine.update_community_baselines()
print("Community baselines initialized")
```
---
## Rollback Procedure
If you need to rollback to V3:
### 1. Stop Services
```bash
# Stop WebSocket server
# Stop Prometheus exporter
```
### 2. Rollback Database
**Automatic Rollback:**
```python
from pathlib import Path
from session_buddy.storage.migrations.base import MigrationManager
manager = MigrationManager(db_path="~/.claude/data/session_buddy.db")
manager.rollback()
# Output:
# Applying migration V4__phase4_extensions__down.sql
# ✓ Dropped 6 views
# ✓ Dropped 14 tables
# ✓ Dropped 3 triggers
# Rollback to V3 complete
```
**Manual Rollback:**
```bash
# Apply rollback SQL
sqlite3 ~/.claude/data/session_buddy.db < session_buddy/storage/migrations/V4__phase4_extensions__down.sql
```
### 3. Restore Backup (Optional)
```bash
# If rollback fails, restore from backup
cp ~/.claude/data/session_buddy.db.v3.backup ~/.claude/data/session_buddy.db
```
---
## Verification
### Test V3 Functionality Still Works
```python
# Test basic V3 operations
from session_buddy.storage.skills_storage import SkillsStorage
storage = SkillsStorage()
# V3 methods still work
session_id = storage.create_session(project_path="/test")
invocation_id = storage.track_invocation(
session_id=session_id,
skill_name="test-skill",
completed=True
)
metrics = storage.get_skill_metrics("test-skill")
print("✓ V3 functionality verified")
```
### Test V4 New Features
```python
# Test V4 real-time metrics
metrics = storage.get_real_time_metrics(limit=5)
print(f"✓ Real-time metrics: {len(metrics)} skills")
# Test anomaly detection
anomalies = storage.detect_anomalies(threshold=2.0)
print(f"✓ Anomaly detection: {len(anomalies)} anomalies")
# Test collaborative filtering
from session_buddy.analytics import get_collaborative_engine
engine = get_collaborative_engine()
recommendations = engine.recommend_from_similar_users("test_user", limit=5)
print(f"✓ Collaborative filtering: {len(recommendations)} recommendations")
```
---
## Performance Considerations
### Database Size Impact
**Expected Growth:**
- Real-time cache: ~1KB per active skill
- Time-series: ~100 bytes per skill-hour (24h = ~2.4KB per skill)
- Anomalies: ~200 bytes per detected anomaly
- Community baselines: ~500 bytes per skill
- User interactions: ~300 bytes per invocation
**Example:** 100 skills, 1000 invocations/day
- V3: ~5MB
- V4: ~15MB (3x increase, primarily time-series data)
### Query Performance
**V4 Views Performance:**
| View | Latency | Notes |
|------|---------|-------|
| `v_realtime_skill_dashboard` | < 50ms | Indexed on skill_name |
| `v_skill_effectiveness_trend` | < 100ms | 7-day window |
| `v_community_baseline_comparison` | < 50ms | Simple JOIN |
| `v_skill_dependency_network` | < 100ms | Lift score calculation |
| `v_ab_test_summary` | < 50ms | Pre-aggregated |
| `v_multimodal_skill_catalog` | < 50ms | Category JOIN |
**Optimization Tips:**
- Time-series queries: Use `(skill_name, timestamp DESC)` index
- Anomaly detection: Limit `time_window_hours` for faster scans
- Collaborative filtering: Cache similar users (TTL: 1 hour)
---
## Troubleshooting
### Issue: Migration Fails
**Symptom:** Error applying V4__up.sql
**Solutions:**
1. **Check database is not locked:**
```bash
# Ensure no processes using the database
lsof ~/.claude/data/session_buddy.db
```
2. **Verify V3 migration was applied:**
```sql
SELECT * FROM skill_migrations ORDER BY version DESC LIMIT 5;
-- Should see V3, V2, V1 entries
```
3. **Check disk space:**
```bash
df -h ~/.claude/data/
# Need at least 100MB free
```
### Issue: WebSocket Server Won't Start
**Symptom:** "Address already in use" error
**Solutions:**
1. **Check port 8765 is free:**
```bash
lsof -i :8765
# Kill existing process if needed
```
2. **Use different port:**
```python
server = RealTimeMetricsServer(port=8766) # Alternate port
```
### Issue: Predictive Model Accuracy Low
**Symptom:** Predictions seem random or inaccurate
**Solutions:**
1. **Ensure sufficient training data:**
```python
predictor = get_predictor("skills.db")
result = predictor.train_model()
print(f"Training samples: {result['samples']}")
# Need at least 1000 invocations for decent accuracy
```
2. **Check feature importance:**
```python
print(result['feature_importance'])
# Features with importance < 0.05 may not be predictive
```
3. **Collect more data:** Model improves with more historical invocations
---
## Next Steps
After successful migration:
1. **Explore New Features:**
- Start WebSocket server for real-time dashboards
- Try collaborative filtering recommendations
- Set up A/B tests for recommendation strategies
2. **Configure Integrations:**
- Enable Crackerjack quality gate tracking
- Set up IDE plugin for context-aware recommendations
- Configure CI/CD pipeline tracking
3. **Monitor Performance:**
- Check anomaly detection reports
- Review time-series trends
- Analyze community baselines
4. **Optimize Over Time:**
- Tune predictive models with more data
- Adjust anomaly detection thresholds
- Refine skill taxonomy based on usage patterns
---
## Support
**Documentation:**
- [V4 Schema Summary](PHASE4_V4_SCHEMA_SUMMARY.md)
- [Wave 1 Completion](PHASE4_WAVE1_COMPLETION_REPORT.md)
- [Wave 2 Completion](PHASE4_WAVE2_COMPLETION_REPORT.md)
- [Wave 3 Completion](PHASE4_WAVE3_COMPLETION_REPORT.md)
- [Final Status](PHASE4_FINAL_STATUS_REPORT.md)
**Testing:**
```bash
# Run Phase 4 integration tests
pytest tests/test_phase4_integration.py -v
# Run with coverage
pytest tests/test_phase4_integration.py --cov=session_buddy --cov-report=html
```
**Migration Status:** ✅ Production Ready
**Rollback Tested:** ✅ Verified
**Breaking Changes:** None
**Recommendation:** Migrate at your convenience
---
**Last Updated:** 2026-02-10
**Migration Version:** V4 (Phase 4: Advanced Analytics & Integration)
