memory-consolidation-guide.md•6.19 kB
# Memory Consolidation System - Operational Guide
**Version**: 8.23.0+ | **Last Updated**: 2025-11-11 | **Status**: Production Ready
## Quick Reference
### System Status Check
```bash
# Check scheduler status
curl http://127.0.0.1:8000/api/consolidation/status
# Verify HTTP server running
systemctl --user status mcp-memory-http.service
```
### Manual Trigger (HTTP API)
```bash
curl -X POST http://127.0.0.1:8000/api/consolidation/trigger \
-H "Content-Type: application/json" \
-d '{"time_horizon": "weekly", "immediate": true}'
```
## Real-World Performance (v8.23.1 Test Results)
**Test Environment**: 2,495 memories, Hybrid backend (SQLite-vec + Cloudflare)
| Backend | First Run | Why |
|---------|-----------|-----|
| **SQLite-Vec** | 5-25s | Local-only, fast |
| **Cloudflare** | 2-4min | Network-dependent |
| **Hybrid** | **4-6min** | Local (~5ms) + Cloud sync (~150ms/update) |
**Key Finding**: Hybrid backend takes longer but provides multi-device data persistence - recommended for production.
## Report Generation Behavior ⚠️
**IMPORTANT**: Reports are only generated when consolidation **COMPLETES**.
### Report Location
```bash
~/.local/share/mcp-memory-service/consolidation/reports/
```
### When Reports Are Created
✅ **After successful consolidation completion**
- Directory is created automatically on first completed consolidation
- Report naming: `consolidation_{horizon}_{timestamp}.json`
❌ **NOT created when**:
- Consolidation is interrupted (killed curl, server restart)
- Consolidation fails
- Consolidation is still running
- No consolidations have run yet
### Verify Reports
```bash
# Check if any consolidations have completed
curl http://127.0.0.1:8000/api/consolidation/status | jq '.jobs_executed'
# If jobs_executed > 0, reports should exist:
ls -lh ~/.local/share/mcp-memory-service/consolidation/reports/
```
**Example**:
- `jobs_executed: 0` = No reports yet (waiting for first scheduled run)
- `jobs_executed: 3` = 3 reports should exist in directory
## Automatic Scheduling
| When | Next Run |
|------|----------|
| **Daily** 02:00 | Processes recent memories |
| **Weekly** Sun 03:00 | Pattern discovery, associations |
| **Monthly** 1st 04:00 | Long-term consolidation, archival |
### Monitor First Scheduled Run
```bash
# Watch logs after first scheduled consolidation (2025-11-12 02:00)
journalctl --user -u mcp-memory-http.service --since "2025-11-12 01:55:00" | grep consolidation
# Then check for reports
ls -lh ~/.local/share/mcp-memory-service/consolidation/reports/
```
## Three Manual Trigger Methods
### 1. HTTP API (Fastest)
```bash
curl -X POST http://127.0.0.1:8000/api/consolidation/trigger \
-H "Content-Type: application/json" \
-d '{"time_horizon": "daily", "immediate": true}'
```
### 2. MCP Tools
```python
mcp__memory__trigger_consolidation(time_horizon="daily", immediate=true)
```
### 3. Code Execution API (Most Token-Efficient)
```python
from mcp_memory_service.api import consolidate
consolidate('daily') # 90% token reduction vs MCP tools
```
**Tip**: Use `daily` for faster test runs (fewer memories to process).
## Monitoring Consolidation
### Real-Time Progress
```bash
journalctl --user -u mcp-memory-http.service -f | grep consolidation
```
### Expected Log Patterns (Hybrid Backend)
```
INFO - Starting weekly consolidation...
INFO - Processing 2495 memories...
INFO - Successfully updated memory metadata: 735d2920...
INFO - HTTP Request: POST https://api.cloudflare.com/.../query "200 OK"
... (repeats for each memory)
INFO - Weekly consolidation completed in 245.3 seconds
INFO - Report saved: consolidation_weekly_2025-11-12_02-00-00.json
```
### Check Completion
```bash
# Method 1: Check jobs_executed counter
curl http://127.0.0.1:8000/api/consolidation/status | jq '.jobs_executed'
# Method 2: Check for report files
ls -lt ~/.local/share/mcp-memory-service/consolidation/reports/ | head -5
```
## Troubleshooting
### No Reports Generated
**Check 1**: Has any consolidation completed?
```bash
curl http://127.0.0.1:8000/api/consolidation/status | jq '.jobs_executed, .jobs_failed'
```
**If `jobs_executed: 0`**:
- No consolidations have completed yet
- Directory won't exist until first completion
- Wait for scheduled run or manually trigger shorter test
**If `jobs_failed > 0`**:
- Check server logs for errors:
```bash
journalctl --user -u mcp-memory-http.service | grep -i "consolidation.*error\|consolidation.*fail"
```
### Consolidation Takes Too Long
**Expected behavior with Hybrid backend**:
- First run: 4-6 minutes (2,495 memories)
- Cloudflare sync adds ~150ms per memory update
- This is normal - provides multi-device persistence
**To speed up**:
- Switch to SQLite-only backend (loses cloud sync)
- Use `daily` time horizon for testing (fewer memories)
### Test Consolidation Completion
**Quick test** (processes fewer memories):
```bash
# Trigger daily consolidation (faster)
curl -X POST http://127.0.0.1:8000/api/consolidation/trigger \
-H "Content-Type: application/json" \
-d '{"time_horizon": "daily", "immediate": true}'
# Wait for completion (watch logs)
journalctl --user -u mcp-memory-http.service -f | grep "consolidation completed"
# Verify report created
ls -lh ~/.local/share/mcp-memory-service/consolidation/reports/
```
## Configuration
### Enable/Disable (.env)
```bash
MCP_CONSOLIDATION_ENABLED=true
MCP_HTTP_ENABLED=true
```
### Schedule (config.py)
```python
CONSOLIDATION_SCHEDULE = {
'daily': '02:00',
'weekly': 'SUN 03:00',
'monthly': '01 04:00',
'quarterly': 'disabled',
'yearly': 'disabled'
}
```
## Summary
- ✅ Consolidation runs automatically (no manual intervention needed)
- ✅ Reports generated only after SUCCESSFUL completion
- ✅ Hybrid backend: 4-6 min first run (normal, provides multi-device sync)
- ✅ `jobs_executed: 0` until first consolidation completes
- ✅ Directory created automatically on first report
- ✅ Monitor scheduled runs via logs and status endpoint
---
**Related**: [Code Execution API](../api/code-execution-interface.md) | [Memory Maintenance](../maintenance/memory-maintenance.md) | [HTTP Server Management](../http-server-management.md)