# Status Translation Backward Compatibility Fix
**Date**: 2025-10-10
**Issue**: Status value incompatibility between Feature 003 work items and legacy task Pydantic models
**Fix Location**: `src/services/tasks.py`
**Test Coverage**: `tests/unit/test_status_translation.py`
## Problem Statement
After Feature 003 introduced polymorphic work items with new status values (`active`, `completed`, `blocked`), the `list_tasks` function began failing because:
1. **Database contains mixed status values**: Both old legacy statuses (`need to be done`, `in-progress`, `complete`) and new Feature 003 statuses (`active`, `completed`, `blocked`)
2. **Pydantic validation is too strict**: `TaskSummary.status` field uses `Literal["need to be done", "in-progress", "complete"]` which rejects any status value not in this exact set
3. **Validation failure at line 396**: Direct Pydantic validation without status translation:
```python
return [TaskSummary.model_validate(task) for task in tasks] # FAILS for new statuses
```
4. **Error message**: `pydantic.ValidationError: Input should be 'need to be done', 'in-progress' or 'complete'`
## Root Cause Analysis
The backward compatibility issue arose from:
1. **Feature 003 changes**: Extended work items with new status vocabulary aligned with project/session/research semantics
2. **Pydantic type constraints**: TaskSummary inherits from BaseTaskFields which defines status as a Literal type (line 85 in `src/models/task_schemas.py`)
3. **No translation layer**: SQLAlchemy models returned from database queries had status values that Pydantic validation rejected
## Solution Architecture
### 1. Status Translation Mapping (Line 56-65)
```python
STATUS_TRANSLATION: Final[dict[str, str]] = {
"active": "need to be done", # New active status → legacy todo
"in-progress": "in-progress", # Same in both systems (no translation)
"completed": "complete", # New completed status → legacy complete
"blocked": "need to be done", # Blocked items shown as todo
"need to be done": "need to be done", # Pass-through old values
"complete": "complete", # Pass-through old values
}
```
**Design Rationale**:
- **Bidirectional compatibility**: Supports both old and new status values
- **Semantic mapping**: `active` → `need to be done` preserves user intent
- **Defensive pass-through**: Legacy values pass unchanged for existing data
- **Blocked handling**: Treats blocked items as "need to be done" (actionable)
### 2. Translation Helper Function (Line 119-159)
```python
def _translate_task_status(task: Task) -> Task:
"""Translate work item status to legacy task status for backward compatibility.
This translation layer is CRITICAL for backward compatibility because:
1. Database now contains both old and new status values after Feature 003
2. TaskSummary.status field uses Literal["need to be done", "in-progress", "complete"]
3. Pydantic validation fails if status doesn't match the literal type
4. This function ensures all status values are translated before validation
Note:
- This is a NON-DESTRUCTIVE operation - modifies in-memory object only
- Does NOT update database - translation is for Pydantic validation only
- If status not in translation map, leaves as-is (defensive behavior)
"""
if task.status in STATUS_TRANSLATION:
task.status = STATUS_TRANSLATION[task.status]
return task
```
**Key Properties**:
- **Non-destructive**: Only modifies in-memory SQLAlchemy object, never touches database
- **Type-safe**: Maintains mypy --strict compliance (returns Task type)
- **Idempotent**: Can be called multiple times safely (pass-through for already-translated values)
- **Defensive**: Leaves status unchanged if not in translation map
### 3. Integration Points
#### `list_tasks()` function (Line 451-462)
```python
# Apply status translation for backward compatibility with Pydantic validation
translated_tasks = [_translate_task_status(task) for task in tasks]
# Conditional serialization
if full_details:
return [_task_to_response(task) for task in translated_tasks]
else:
return [TaskSummary.model_validate(task) for task in translated_tasks]
```
#### `get_task()` function (Line 353-357)
```python
# Apply status translation for backward compatibility with Pydantic validation
task = _translate_task_status(task)
return _task_to_response(task)
```
## Test Coverage
### Unit Tests (`tests/unit/test_status_translation.py`)
Comprehensive test suite with 9 test cases covering:
1. ✅ **Status translation mapping**: All 6 status values (active, completed, blocked, in-progress, need to be done, complete)
2. ✅ **Pydantic validation success**: TaskSummary validates after translation
3. ✅ **Pydantic validation failure**: Demonstrates bug exists without translation
4. ✅ **Non-destructive behavior**: In-memory modification only
5. ✅ **Complete coverage**: All STATUS_TRANSLATION entries tested
**Test Results**:
```
tests/unit/test_status_translation.py::TestStatusTranslation::test_translate_active_to_need_to_be_done PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_translate_completed_to_complete PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_translate_blocked_to_need_to_be_done PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_translate_inprogress_passthrough PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_translate_legacy_statuses_passthrough PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_pydantic_validation_succeeds_after_translation PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_pydantic_validation_fails_without_translation PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_translation_is_non_destructive PASSED
tests/unit/test_status_translation.py::TestStatusTranslation::test_all_status_translations_coverage PASSED
9 passed in 1.34s
```
## Type Safety Validation
**mypy --strict compliance**: ✅ Zero type errors
```bash
$ python -m mypy src/services/tasks.py --strict
Success: no issues found in 1 source file
```
## Constitutional Compliance
This fix adheres to project constitutional principles:
1. **Principle V: Production Quality**
- Maintains 100% backward compatibility
- Comprehensive error handling
- No database modifications (non-destructive)
2. **Principle VIII: Type Safety**
- Full mypy --strict compliance
- Explicit type annotations
- Pydantic validation preserved
3. **Principle VII: Test-Driven Development**
- Comprehensive unit test coverage (9 tests)
- Tests demonstrate bug and validate fix
- Regression prevention
## Impact Analysis
### What This Fix Does
- ✅ Enables `list_tasks()` to work with mixed status values
- ✅ Enables `get_task()` to work with mixed status values
- ✅ Maintains 100% backward compatibility
- ✅ Zero database modifications
- ✅ Preserves type safety (mypy --strict)
### What This Fix Does NOT Do
- ❌ Does NOT modify database schema
- ❌ Does NOT change status values in database
- ❌ Does NOT affect `create_task()` or `update_task()` operations
- ❌ Does NOT change Pydantic model definitions
## Future Considerations
### Long-Term Solution Options
**Option 1: Update Pydantic Models (Recommended)**
```python
# In src/models/task_schemas.py
status: Literal[
"need to be done", "in-progress", "complete", # Legacy
"active", "completed", "blocked" # Feature 003
]
```
**Pros**:
- Eliminates need for translation layer
- More transparent status handling
- Better aligns with database reality
**Cons**:
- Requires updating all downstream consumers
- May break existing API contracts
- Requires coordinated deployment
**Option 2: Database Migration (Not Recommended)**
```sql
-- Migrate all new statuses to legacy statuses
UPDATE tasks SET status = 'need to be done' WHERE status = 'active';
UPDATE tasks SET status = 'complete' WHERE status = 'completed';
```
**Pros**:
- Eliminates status vocabulary mismatch
- Simplifies codebase
**Cons**:
- Loses semantic information from Feature 003
- Breaks Feature 003 work item semantics
- Requires data migration
- May affect other features
**Recommendation**: Keep current translation layer for backward compatibility, plan migration to Option 1 in a future feature when API contracts can be updated.
## Files Modified
1. **`src/services/tasks.py`**
- Added `STATUS_TRANSLATION` constant (line 56-65)
- Added `_translate_task_status()` helper function (line 119-159)
- Modified `list_tasks()` to apply translation (line 451-462)
- Modified `get_task()` to apply translation (line 353-357)
2. **`tests/unit/test_status_translation.py`** (NEW)
- Comprehensive unit test suite (9 test cases)
- Validates translation mapping correctness
- Demonstrates Pydantic validation fix
3. **`docs/2025-10-10-status-translation-fix.md`** (THIS FILE)
- Implementation documentation
- Architectural decisions
- Future considerations
## Verification Steps
To verify this fix is working correctly:
1. **Run unit tests**:
```bash
python -m pytest tests/unit/test_status_translation.py -v
```
2. **Verify type safety**:
```bash
python -m mypy src/services/tasks.py --strict
```
3. **Test with mixed database data**:
```python
# Create work item with new status
work_item = WorkItem(
item_type="task",
title="Test task",
status="active", # New status
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc)
)
db.add(work_item)
await db.commit()
# List tasks should work without error
tasks = await list_tasks(db, limit=10)
assert len(tasks) > 0
assert tasks[0].status in ["need to be done", "in-progress", "complete"]
```
## Summary
This fix implements a minimal, non-destructive status translation layer that:
- ✅ Solves the immediate Pydantic validation failure
- ✅ Maintains 100% backward compatibility
- ✅ Preserves type safety (mypy --strict compliance)
- ✅ Has comprehensive test coverage (9 unit tests)
- ✅ Requires no database changes
- ✅ Aligns with constitutional principles
The translation approach is a pragmatic short-term solution that enables Feature 003 work items to coexist with legacy task Pydantic models while maintaining production quality and type safety standards.
