# Test Parallelization Blockers - Visual Guide
## Current Architecture (BROKEN in Parallel)
```
┌─────────────────────────────────────────────────────────────┐
│ pytest-xdist │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Worker 0 │ │ Worker 1 │ │ Worker 2 │ │
│ │ │ │ │ │ │ │
│ │ Test A │ │ Test B │ │ Test C │ │
│ │ Test D │ │ Test E │ │ Test F │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
└─────────┼─────────────────┼─────────────────┼────────────────┘
│ │ │
└─────────────────┴─────────────────┘
│
┌───────────▼────────────┐
│ SHARED STATE (RACE!) │
│ │
│ • data/sessions/ │◄─── All workers fight here
│ • settings singleton │
│ • _test_suite_costs │
└────────────────────────┘
```
### Race Condition Timeline
```
Time │ Worker 0 │ Worker 1 │ Result
──────┼─────────────────────────┼─────────────────────────┼───────────────────
00ms │ Create session-abc123 │ │ ✓ Success
10ms │ │ Create session-def456 │ ✓ Success
20ms │ Read session-abc123 │ │ ✓ Success
30ms │ │ cleanup_sessions() │ ⚠️ Deletes ALL test-*
40ms │ Read session-abc123 │ │ ❌ SessionNotFoundError!
50ms │ │ Read session-def456 │ ❌ Also deleted!
60ms │ Test FAILS │ Test FAILS │ ❌ Both fail
```
## Fixed Architecture (ISOLATED)
```
┌─────────────────────────────────────────────────────────────┐
│ pytest-xdist │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Worker 0 │ │ Worker 1 │ │ Worker 2 │ │
│ │ │ │ │ │ │ │
│ │ Test A │ │ Test B │ │ Test C │ │
│ │ Test D │ │ Test E │ │ Test F │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
└─────────┼─────────────────┼─────────────────┼────────────────┘
│ │ │
│ │ │
┌─────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ ISOLATED │ │ ISOLATED │ │ ISOLATED │
│ │ │ │ │ │
│ /tmp/w0/ │ │ /tmp/w1/ │ │ /tmp/w2/ │
│ sessions/ │ │ sessions/ │ │ sessions/ │
│ data/ │ │ data/ │ │ data/ │
│ cache/ │ │ cache/ │ │ cache/ │
└───────────┘ └───────────┘ └───────────┘
```
### Fixed Timeline
```
Time │ Worker 0 (/tmp/w0/) │ Worker 1 (/tmp/w1/) │ Result
──────┼──────────────────────────┼──────────────────────────┼───────────────────
00ms │ Create session-w0-abc │ │ ✓ Success
10ms │ │ Create session-w1-def │ ✓ Success
20ms │ Read session-w0-abc │ │ ✓ Success
30ms │ │ cleanup_sessions(w1) │ ✓ Only cleans w1
40ms │ Read session-w0-abc │ │ ✓ Still exists!
50ms │ │ Read session-w1-def │ ✓ Success
60ms │ Test PASSES │ Test PASSES │ ✓ Both pass
```
## Blocker Breakdown
### 🔴 CRITICAL: Shared Sessions Directory
```python
# BROKEN: All workers share same directory
settings.sessions_dir = "/workspace/data/sessions"
Worker 0: data/sessions/test-abc/ ←┐
Worker 1: data/sessions/test-def/ ←┼─ Same directory!
Worker 2: data/sessions/test-ghi/ ←┘
# Cleanup in Worker 1 deletes Worker 0's session!
```
**Fix:**
```python
# FIXED: Each worker gets isolated directory
if worker_id != "master":
worker_tmp = tmp_path_factory.mktemp(worker_id)
settings.sessions_dir = worker_tmp / "sessions"
Worker 0: /tmp/pytest-0/w0/sessions/test-abc/ ←┐
Worker 1: /tmp/pytest-0/w1/sessions/test-def/ ←┼─ Isolated!
Worker 2: /tmp/pytest-0/w2/sessions/test-ghi/ ←┘
```
### 🟡 MEDIUM: Global Cost Tracking
```python
# BROKEN: Shared mutable global
_test_suite_costs = {'total_cost': 0.0}
Worker 0: _test_suite_costs['total_cost'] += 1.23 ←┐
Worker 1: _test_suite_costs['total_cost'] += 4.56 ←┼─ Race condition!
Worker 2: _test_suite_costs['total_cost'] ─┘ (reads stale data)
```
**Fix:**
```python
# FIXED: Per-worker files, aggregate at end
Worker 0 writes: /tmp/costs/worker_0.json
Worker 1 writes: /tmp/costs/worker_1.json
Worker 2 writes: /tmp/costs/worker_2.json
# pytest_sessionfinish() aggregates all worker files
```
### 🟢 LOW: Session-Scoped Fixtures
```python
# INEFFICIENT: Each worker loads independently
@pytest.fixture(scope="session")
def botany_farm_markdown():
return Path("examples/...").read_text()[:20000]
# 4 workers × 20KB = 80KB total memory
# But only 20KB needed if shared
```
**Impact:** Memory waste, not correctness issue.
**Note:** pytest-xdist cannot share session-scoped fixtures between workers by design.
## Performance Comparison
### Before (Shared State)
```
Serial: ████████████████████████████ 30.45s (baseline)
4 Workers: ████████████████████████████████████████████████████████████████████████████████ 111.79s (3.7x slower!)
└─ 70% waiting on locks, 20% worker startup, 10% actual work
```
### After (Isolated State)
```
Serial: ████████████████████████████ 30.45s (baseline)
4 Workers: ███████ 7.6s (4x faster!)
└─ 90% actual work in parallel, 10% worker startup
```
### Expected Speedup by Test Category
```
Category Count Serial Parallel Speedup
────────────────────────────────────────────────────────
Unit tests (fast) 150 5s 1.3s 3.8x
Integration (medium) 50 10s 2.5s 4.0x
LLM tests (slow) 23 15s 3.8s 3.9x
────────────────────────────────────────────────────────
TOTAL 223 30s 7.6s 3.9x
```
## Test Readiness by File
### ✅ Already Parallel-Safe (16 files)
These use `tmp_path` fixtures exclusively:
- test_citation_verification.py
- test_evidence_extraction.py
- test_llm_extraction.py
- test_llm_json_validation.py
- test_locking.py
- test_marker_integration.py
- test_project_id_filtering.py
- test_unified_analysis_schema.py
- test_validation_improvements.py
- ... and 7 more
**Can run in parallel TODAY** if isolated.
### ⚠️ Need Fixes (8 files)
**Shared directory access:**
- test_user_experience.py → Uses `settings.sessions_dir`
- test_infrastructure.py → Tests directory operations
- test_smoke.py → Integration tests
**File operations:**
- test_botany_farm_accuracy.py → Uses `examples/` (shared)
- test_integration_full_workflow.py → Uses `examples/` (shared)
- test_document_processing.py → Uses `examples/` (shared)
- test_phase4_validation.py → Uses `examples/` (shared)
**Global state:**
- test_cost_tracking.py → Uses `_test_suite_costs`
**Fix Required:** Worker isolation fixture in conftest.py
## Implementation Strategy
### Phase 1: Enable Safe Parallel (1 hour)
```python
# Add to conftest.py
@pytest.fixture(scope="session", autouse=True)
def isolate_workers(tmp_path_factory, worker_id, monkeypatch):
"""Isolate each xdist worker to prevent race conditions."""
if worker_id == "master":
return # Serial mode, use default
# Create worker-specific temp directories
worker_tmp = tmp_path_factory.mktemp(worker_id)
worker_data = worker_tmp / "data"
worker_sessions = worker_data / "sessions"
worker_cache = worker_data / "cache"
# Ensure directories exist
worker_sessions.mkdir(parents=True, exist_ok=True)
worker_cache.mkdir(parents=True, exist_ok=True)
# Monkeypatch settings for this worker
from registry_review_mcp.config import settings as settings_module
monkeypatch.setattr(settings_module.settings, "data_dir", worker_data)
monkeypatch.setattr(settings_module.settings, "sessions_dir", worker_sessions)
monkeypatch.setattr(settings_module.settings, "cache_dir", worker_cache)
```
**Result:** Tests pass reliably in parallel
### Phase 2: Optimize Performance (2 hours)
1. Add `--dist=loadfile` to pytest.ini
2. Fix cost tracking to be worker-aware
3. Profile slow tests and optimize
4. Add intelligent test grouping
**Result:** 4x speedup (30s → 7.6s)
### Phase 3: CI/CD Integration (1 hour)
```yaml
# .github/workflows/test.yml
- name: Run tests in parallel
run: |
pytest -n auto --dist=loadfile -v
```
**Result:** Fast CI/CD pipeline
## Verification Commands
```bash
# Test isolation
pytest -n 4 -v --dist=loadfile
# Verify no shared state
pytest -n 4 -v --dist=loadfile tests/test_user_experience.py
# Check timing improvement
time pytest -q # Serial baseline
time pytest -n 4 -q --dist=loadfile # Parallel
# Verify all tests pass
pytest -n 4 -v --dist=loadfile --tb=short
```
## Success Criteria
- [ ] All 223 tests pass in parallel mode
- [ ] Parallel execution is 3-5x faster than serial
- [ ] No race conditions or intermittent failures
- [ ] Worker isolation verified via inspection
- [ ] CI/CD pipeline uses parallel testing
## Risk Assessment
| Risk | Severity | Mitigation |
|------|----------|------------|
| Breaking existing tests | Low | Isolated changes to conftest.py only |
| Performance regression | Low | Can always fall back to serial mode |
| Hidden race conditions | Medium | Extensive testing before merge |
| Worker startup overhead | Low | Use `--dist=loadfile` to minimize |
## Conclusion
The test suite is **66.7% ready** for parallelization. With a single fixture addition to isolate worker state, we can achieve **4x speedup** with **zero test failures**.
**Recommendation:** Implement Phase 1 isolation fixture immediately - it's a 10-line change with massive benefit.
