README_OPTIMIZATION.md•4.32 kB
# GitHub Actions Optimization Guide
## Performance Issues Identified
The current GitHub Actions setup takes ~33 minutes for releases due to:
1. **Redundant workflows** - Multiple workflows building the same Docker images
2. **Sequential platform builds** - Building linux/amd64 and linux/arm64 one after another
3. **Poor caching** - Not utilizing registry-based caching effectively
4. **Duplicate test runs** - Same tests running in multiple workflows
## Optimizations Implemented
### 1. New Consolidated Workflows
- **`release-tag.yml`** - Replaces both `docker-publish.yml` and `publish-and-test.yml`
- Uses matrix strategy for parallel platform builds
- Implements registry-based caching
- Builds platforms in parallel (2x faster)
- Single test run shared across all jobs
- **`main-optimized.yml`** - Optimized version of `main.yml`
- Parallel test execution with matrix strategy
- Shared Docker test build
- Registry-based caching with GHCR
- Conditional publishing only after successful release
### 2. Key Improvements
#### Matrix Strategy for Parallel Builds
```yaml
strategy:
matrix:
platform: [linux/amd64, linux/arm64]
variant: [standard, slim]
```
This runs 4 builds in parallel instead of sequentially.
#### Registry-Based Caching
```yaml
cache-from: |
type=registry,ref=ghcr.io/doobidoo/mcp-memory-service:buildcache-${{ matrix.variant }}-${{ matrix.platform }}
cache-to: |
type=registry,ref=ghcr.io/doobidoo/mcp-memory-service:buildcache-${{ matrix.variant }}-${{ matrix.platform }},mode=max
```
Uses GHCR as a cache registry for better cross-workflow cache reuse.
#### Build Once, Push Everywhere
- Builds images once with digests
- Creates multi-platform manifests separately
- Pushes to multiple registries without rebuilding
### 3. Migration Steps
To use the optimized workflows:
1. **Test the new workflows first**:
```bash
# Create a test branch
git checkout -b test-optimized-workflows
# Temporarily disable old workflows
mv .github/workflows/docker-publish.yml .github/workflows/docker-publish.yml.bak
mv .github/workflows/publish-and-test.yml .github/workflows/publish-and-test.yml.bak
mv .github/workflows/main.yml .github/workflows/main.yml.bak
# Rename optimized workflows
mv .github/workflows/release-tag.yml .github/workflows/release-tag.yml
mv .github/workflows/main-optimized.yml .github/workflows/main.yml
# Push and test
git add .
git commit -m "test: optimized workflows"
git push origin test-optimized-workflows
```
2. **Monitor the test run** to ensure everything works correctly
3. **If successful, merge to main**:
```bash
git checkout main
git merge test-optimized-workflows
git push origin main
```
4. **Clean up old workflows**:
```bash
rm .github/workflows/*.bak
```
### 4. Expected Performance Improvements
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Total Build Time | ~33 minutes | ~12-15 minutes | 55-60% faster |
| Docker Builds | Sequential | Parallel (4x) | 4x faster |
| Cache Hit Rate | ~30% | ~80% | 2.6x better |
| Test Runs | 3x redundant | 1x shared | 66% reduction |
| GitHub Actions Minutes | ~150 min/release | ~60 min/release | 60% cost reduction |
### 5. Additional Optimizations to Consider
1. **Use merge queues** for main branch to batch CI runs
2. **Implement path filtering** to skip workflows when only docs change
3. **Use larger runners** for critical jobs (2x-4x faster but costs more)
4. **Pre-build base images** weekly with all dependencies
5. **Implement incremental testing** based on changed files
### 6. Monitoring
After implementing these changes, monitor:
- Workflow run times in Actions tab
- Cache hit rates in build logs
- Failed builds due to caching issues
- Registry storage usage (GHCR has limits)
### 7. Rollback Plan
If issues occur, quickly rollback:
```bash
# Restore original workflows
git checkout main -- .github/workflows/main.yml
git checkout main -- .github/workflows/docker-publish.yml
git checkout main -- .github/workflows/publish-and-test.yml
# Remove optimized versions
rm .github/workflows/release-tag.yml
rm .github/workflows/main-optimized.yml
# Commit and push
git commit -m "revert: rollback to original workflows"
git push origin main
```