# Docker Compose Auto-Discovery Implementation Plan
> **📁 Organization Note:** When this plan is fully implemented and verified, move this file to `docs/plans/complete/` to keep the plans folder organized.
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Enable frictionless Docker Compose operations by automatically discovering and caching compose file locations across all hosts, eliminating the need for users to specify file paths manually.
**Architecture:** Multi-layered discovery system that checks cache first, then running containers (`docker compose ls`), then filesystem scans. Per-host caching with intelligent invalidation. Optional host parameter auto-resolves to unique matches across all hosts. Parallel execution for performance.
**Tech Stack:** TypeScript, Zod schemas, JSON file cache, SSH commands via existing infrastructure, filesystem scanning with `find`
---
## Task 0: Create IComposeProjectLister Interface
**Files:**
- Modify: `src/services/interfaces.ts` (add IComposeProjectLister interface)
- Test: `src/services/interfaces.test.ts` (new file)
**Purpose:** Create foundational interface needed by ComposeDiscovery to avoid circular dependency with ComposeService.
**Note:** We're NOT creating Zod schemas for HostConfig because:
1. HostConfig interface already exists in `src/types.ts`
2. ComposeProject type already exists in `src/services/compose.ts` (we'll import it)
3. Runtime validation via Zod is not needed for this feature (TypeScript interfaces sufficient)
**Step 1: Write the failing test**
```typescript
// src/services/interfaces.test.ts - NEW FILE
import { describe, it, expect } from 'vitest';
import type { IComposeProjectLister } from './interfaces.js';
import type { HostConfig } from '../types.js';
import type { ComposeProject } from './compose.js';
describe('IComposeProjectLister', () => {
it('should be implemented with listComposeProjects method', async () => {
const mockLister: IComposeProjectLister = {
listComposeProjects: async (host: HostConfig): Promise<ComposeProject[]> => {
return [
{
name: 'test-project',
status: 'running',
configFiles: ['/compose/test/docker-compose.yaml'],
services: []
}
];
}
};
const host: HostConfig = {
name: 'test',
host: 'localhost',
protocol: 'ssh'
};
const result = await mockLister.listComposeProjects(host);
expect(result).toHaveLength(1);
expect(result[0].name).toBe('test-project');
expect(result[0].configFiles).toContain('/compose/test/docker-compose.yaml');
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/services/interfaces.test.ts`
Expected: FAIL with "Cannot find type 'IComposeProjectLister'"
**Step 3: Add IComposeProjectLister interface to existing interfaces.ts**
```typescript
// src/services/interfaces.ts - ADD this interface to the end of the file
/**
* Minimal interface for listing compose projects
* Used by ComposeDiscovery to avoid circular dependency with ComposeService
*/
export interface IComposeProjectLister {
listComposeProjects(host: HostConfig): Promise<ComposeProject[]>;
}
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/services/interfaces.test.ts`
Expected: PASS
**Step 5: Verify no syntax errors**
Run: `pnpm run typecheck`
Expected: No errors
**Step 6: Commit**
```bash
git add src/services/interfaces.ts src/services/interfaces.test.ts
git commit -m "feat: add IComposeProjectLister interface with TDD"
```
---
## Task 1: Add Configuration Schema for Custom Search Paths
**Files:**
- Modify: `src/types.ts` (add composeSearchPaths to HostConfig)
- Test: `src/types.test.ts` (new file)
**Step 1: Write the failing test**
```typescript
// src/types.test.ts - NEW FILE
import { describe, it, expect } from 'vitest';
import type { HostConfig } from './types.js';
describe('HostConfig', () => {
it('should support optional composeSearchPaths field', () => {
const config: HostConfig = {
name: 'test',
host: 'localhost',
protocol: 'ssh',
composeSearchPaths: ['/opt/stacks', '/srv/docker']
};
expect(config.composeSearchPaths).toEqual(['/opt/stacks', '/srv/docker']);
});
it('should work without composeSearchPaths', () => {
const config: HostConfig = {
name: 'test',
host: 'localhost',
protocol: 'ssh'
};
expect(config.composeSearchPaths).toBeUndefined();
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/types.test.ts`
Expected: FAIL with "composeSearchPaths does not exist in type HostConfig"
**Step 3: Add composeSearchPaths to HostConfig interface**
```typescript
// src/types.ts - modify existing HostConfig interface (around line 46)
export interface HostConfig {
name: string;
host: string;
port?: number;
protocol: "http" | "https" | "ssh";
sshUser?: string;
sshKeyPath?: string;
dockerSocketPath?: string;
tags?: string[];
composeSearchPaths?: string[]; // Add this line
}
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/types.test.ts`
Expected: PASS
**Step 5: Commit**
```bash
git add src/types.ts src/types.test.ts
git commit -m "feat: add composeSearchPaths to host configuration"
```
---
## Task 2: Create Compose Project Cache Data Structure
**Files:**
- Create: `src/services/compose-cache.ts`
- Test: `src/services/compose-cache.test.ts`
**Step 1: Write the failing test**
```typescript
// src/services/compose-cache.test.ts
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { rm, mkdir } from 'fs/promises';
import { ComposeProjectCache } from './compose-cache.js';
describe('ComposeProjectCache', () => {
const testCacheDir = '.cache/test-compose-projects';
let cache: ComposeProjectCache;
beforeEach(async () => {
await mkdir(testCacheDir, { recursive: true });
cache = new ComposeProjectCache(testCacheDir);
});
afterEach(async () => {
await rm(testCacheDir, { recursive: true, force: true });
});
it('should load empty cache for new host', async () => {
const data = await cache.load('test-host');
expect(data).toEqual({
lastScan: expect.any(String),
searchPaths: [],
projects: {}
});
});
it('should save and load cache data', async () => {
const data = {
lastScan: new Date().toISOString(),
searchPaths: ['/compose', '/mnt/cache/compose'],
projects: {
plex: {
path: '/mnt/cache/compose/plex/docker-compose.yaml',
name: 'plex',
discoveredFrom: 'docker-ls' as const,
lastSeen: new Date().toISOString()
}
}
};
await cache.save('test-host', data);
const loaded = await cache.load('test-host');
expect(loaded).toEqual(data);
});
it('should get project from cache', async () => {
const data = {
lastScan: new Date().toISOString(),
searchPaths: ['/compose'],
projects: {
plex: {
path: '/mnt/cache/compose/plex/docker-compose.yaml',
name: 'plex',
discoveredFrom: 'docker-ls' as const,
lastSeen: new Date().toISOString()
}
}
};
await cache.save('test-host', data);
const project = await cache.getProject('test-host', 'plex');
expect(project?.path).toBe('/mnt/cache/compose/plex/docker-compose.yaml');
});
it('should return undefined for missing project', async () => {
const project = await cache.getProject('test-host', 'missing');
expect(project).toBeUndefined();
});
// Security: Path traversal protection (CWE-22)
it('should reject host names with path traversal attempts', async () => {
await expect(cache.load('../../../etc')).rejects.toThrow('Invalid');
await expect(cache.load('host/../passwd')).rejects.toThrow('Invalid');
await expect(cache.load('../../secret')).rejects.toThrow('Invalid');
});
it('should reject host names with invalid characters', async () => {
await expect(cache.load('host;rm -rf')).rejects.toThrow('Invalid');
await expect(cache.load('host|cat /etc/passwd')).rejects.toThrow('Invalid');
await expect(cache.load('host$HOME')).rejects.toThrow('Invalid');
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/services/compose-cache.test.ts`
Expected: FAIL with "Cannot find module './compose-cache.js'"
**Step 3: Implement ComposeProjectCache**
```typescript
// src/services/compose-cache.ts
import { readFile, writeFile, mkdir } from 'fs/promises';
import { join } from 'path';
import { validateHostFormat } from '../utils/path-security.js';
export interface CachedProject {
path: string;
name: string;
discoveredFrom: 'docker-ls' | 'scan' | 'user-provided';
lastSeen: string;
}
export interface CacheData {
lastScan: string;
searchPaths: string[];
projects: Record<string, CachedProject>;
}
export class ComposeProjectCache {
constructor(private cacheDir = '.cache/compose-projects') {}
/**
* Validates host name to prevent path traversal attacks (CWE-22)
* Reuses existing validateHostFormat from path-security.ts
*/
private validateHost(host: string): void {
validateHostFormat(host);
}
async load(host: string): Promise<CacheData> {
this.validateHost(host);
const file = join(this.cacheDir, `${host}.json`);
try {
const data = await readFile(file, 'utf-8');
return JSON.parse(data);
} catch {
return this.emptyCache();
}
}
async save(host: string, data: CacheData): Promise<void> {
this.validateHost(host);
await mkdir(this.cacheDir, { recursive: true });
const file = join(this.cacheDir, `${host}.json`);
await writeFile(file, JSON.stringify(data, null, 2));
}
async getProject(host: string, projectName: string): Promise<CachedProject | undefined> {
const data = await this.load(host);
return data.projects[projectName];
}
async updateProject(
host: string,
projectName: string,
project: CachedProject
): Promise<void> {
const data = await this.load(host);
data.projects[projectName] = project;
data.lastScan = new Date().toISOString();
await this.save(host, data);
}
async removeProject(host: string, projectName: string): Promise<void> {
const data = await this.load(host);
delete data.projects[projectName];
await this.save(host, data);
}
private emptyCache(): CacheData {
return {
lastScan: new Date().toISOString(),
searchPaths: [],
projects: {}
};
}
}
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/services/compose-cache.test.ts`
Expected: PASS
**Step 5: Commit**
```bash
git add src/services/compose-cache.ts src/services/compose-cache.test.ts
git commit -m "feat: implement compose project cache with file persistence"
```
---
## Task 3: Create Compose File Scanner Service
**Files:**
- Create: `src/services/compose-scanner.ts`
- Test: `src/services/compose-scanner.test.ts`
**Step 0: Install yaml dependency**
Run: `pnpm add yaml`
**Step 1: Write the failing test**
```typescript
// src/services/compose-scanner.test.ts
import { describe, it, expect, vi } from 'vitest';
import { ComposeScanner } from './compose-scanner.js';
import type { ISSHService, ILocalExecutorService } from './interfaces.js';
describe('ComposeScanner', () => {
const mockSSH: ISSHService = {
executeSSHCommand: vi.fn()
} as any;
const mockLocalExecutor: ILocalExecutorService = {
executeLocalCommand: vi.fn()
} as any;
const scanner = new ComposeScanner(mockSSH, mockLocalExecutor);
it('should find compose files via SSH', async () => {
vi.mocked(mockSSH.executeSSHCommand).mockResolvedValue(
'/compose/plex/docker-compose.yaml\n' +
'/mnt/cache/compose/sonarr/docker-compose.yml\n' +
'/mnt/cache/code/nugget/docker-compose.yaml'
);
const host = { name: 'test', host: '192.168.1.1', sshUser: 'user' };
const paths = ['/compose', '/mnt/cache/compose', '/mnt/cache/code'];
const results = await scanner.findComposeFiles(host, paths);
expect(results).toEqual([
'/compose/plex/docker-compose.yaml',
'/mnt/cache/compose/sonarr/docker-compose.yml',
'/mnt/cache/code/nugget/docker-compose.yaml'
]);
});
it('should extract project name from directory', () => {
const name = scanner.extractProjectName('/mnt/cache/compose/plex/docker-compose.yaml');
expect(name).toBe('plex');
});
it('should extract project name from nested compose file', () => {
const name = scanner.extractProjectName('/mnt/cache/code/nugget/docker/compose.yml');
expect(name).toBe('docker');
});
it('should parse compose file for explicit name via SSH', async () => {
const composeContent = `
name: my-custom-name
services:
web:
image: nginx
`;
vi.mocked(mockSSH.executeSSHCommand).mockResolvedValue(composeContent);
const host = { name: 'test', host: '192.168.1.1', sshUser: 'user' };
const name = await scanner.parseComposeName(host, '/compose/app/docker-compose.yaml');
expect(name).toBe('my-custom-name');
});
it('should return null if no explicit name in compose file', async () => {
const composeContent = `
services:
web:
image: nginx
`;
vi.mocked(mockSSH.executeSSHCommand).mockResolvedValue(composeContent);
const host = { name: 'test', host: '192.168.1.1', sshUser: 'user' };
const name = await scanner.parseComposeName(host, '/compose/app/docker-compose.yaml');
expect(name).toBeNull();
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/services/compose-scanner.test.ts`
Expected: FAIL with "Cannot find module './compose-scanner.js'"
**Step 3: Implement ComposeScanner**
```typescript
// src/services/compose-scanner.ts
import { basename, dirname } from 'path';
import { parse as parseYaml } from 'yaml';
import type { HostConfig } from '../types.js';
import { validateHostForSsh } from './ssh.js';
import { isLocalHost } from '../utils/host-utils.js';
import type { ISSHService, ILocalExecutorService } from './interfaces.js';
const DEFAULT_SEARCH_PATHS = ['/compose', '/mnt/cache/compose', '/mnt/cache/code'];
const MAX_SCAN_DEPTH = 3;
export class ComposeScanner {
constructor(
private sshService: ISSHService,
private localExecutor: ILocalExecutorService
) {}
/**
* Find all compose files in the specified search paths
* Uses multiple find calls to avoid shell metacharacters
*/
async findComposeFiles(host: HostConfig, searchPaths: string[]): Promise<string[]> {
const paths = searchPaths.length > 0 ? searchPaths : DEFAULT_SEARCH_PATHS;
const filePatterns = ['docker-compose.yml', 'docker-compose.yaml', 'compose.yml', 'compose.yaml'];
const allFiles: string[] = [];
// Run find for each pattern separately to avoid shell complexity
for (const pattern of filePatterns) {
for (const searchPath of paths) {
try {
let stdout: string;
if (isLocalHost(host)) {
// Use array args to avoid shell injection
stdout = await this.localExecutor.executeLocalCommand(
'find',
[searchPath, '-maxdepth', String(MAX_SCAN_DEPTH), '-type', 'f', '-name', pattern],
{ timeoutMs: 10000 }
);
} else {
validateHostForSsh(host);
// For SSH, use args array to avoid shell injection
// executeSSHCommand will properly escape args internally
const args = [searchPath, '-maxdepth', String(MAX_SCAN_DEPTH), '-type', 'f', '-name', pattern];
stdout = await this.sshService.executeSSHCommand(
host,
'find',
args,
{ timeoutMs: 10000 }
);
}
const files = stdout.trim().split('\n').filter(line => line.length > 0);
allFiles.push(...files);
} catch {
// Ignore errors for individual searches (e.g., path doesn't exist)
}
}
}
// Deduplicate files
return Array.from(new Set(allFiles));
}
/**
* Extract project name from compose file path (parent directory name)
*/
extractProjectName(composePath: string): string {
return basename(dirname(composePath));
}
/**
* Parse compose file to extract explicit 'name:' field
* Returns null if no explicit name is defined
*/
async parseComposeName(host: HostConfig, composePath: string): Promise<string | null> {
try {
let content: string;
if (isLocalHost(host)) {
content = await this.localExecutor.executeLocalCommand('cat', [composePath], {
timeoutMs: 5000
});
} else {
validateHostForSsh(host);
// Use args array to avoid shell injection
content = await this.sshService.executeSSHCommand(
host,
'cat',
[composePath],
{ timeoutMs: 5000 }
);
}
const parsed = parseYaml(content) as { name?: string };
return parsed.name ?? null;
} catch {
return null;
}
}
}
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/services/compose-scanner.test.ts`
Expected: PASS
**Step 5: Commit**
```bash
git add src/services/compose-scanner.ts src/services/compose-scanner.test.ts package.json pnpm-lock.yaml
git commit -m "feat: implement compose file scanner with SSH support"
```
---
## Task 4: Create Compose Discovery Service
**Files:**
- Create: `src/services/compose-discovery.ts`
- Test: `src/services/compose-discovery.test.ts`
**Step 1: Write the failing test**
```typescript
// src/services/compose-discovery.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { ComposeDiscovery } from './compose-discovery.js';
import type { IComposeProjectLister } from './interfaces.js';
import { ComposeProjectCache } from './compose-cache.js';
import { ComposeScanner } from './compose-scanner.js';
describe('ComposeDiscovery', () => {
const mockProjectLister: IComposeProjectLister = {
listComposeProjects: vi.fn()
} as any;
const mockCache = {
load: vi.fn(),
save: vi.fn(),
getProject: vi.fn(),
updateProject: vi.fn(),
removeProject: vi.fn()
} as any;
const mockScanner = {
findComposeFiles: vi.fn(),
extractProjectName: vi.fn(),
parseComposeName: vi.fn()
} as any;
const mockSSH = {
executeSSHCommand: vi.fn()
} as any;
let discovery: ComposeDiscovery;
beforeEach(() => {
vi.clearAllMocks();
discovery = new ComposeDiscovery(mockProjectLister, mockCache, mockScanner, mockSSH);
});
it('should return cached path if found', async () => {
mockCache.getProject.mockResolvedValue({
path: '/compose/plex/docker-compose.yaml',
name: 'plex',
discoveredFrom: 'docker-ls',
lastSeen: new Date().toISOString()
});
const host = { name: 'test', host: 'localhost' };
const result = await discovery.resolveProjectPath(host, 'plex');
expect(result).toBe('/compose/plex/docker-compose.yaml');
expect(mockProjectLister.listComposeProjects).not.toHaveBeenCalled();
});
it('should discover from docker compose ls if not cached', async () => {
mockCache.getProject.mockResolvedValue(undefined);
mockProjectLister.listComposeProjects.mockResolvedValue([
{
name: 'plex',
status: 'running',
configFiles: ['/compose/plex/docker-compose.yaml'],
services: []
}
]);
const host = { name: 'test', host: 'localhost' };
const result = await discovery.resolveProjectPath(host, 'plex');
expect(result).toBe('/compose/plex/docker-compose.yaml');
expect(mockCache.updateProject).toHaveBeenCalledWith(
'test',
'plex',
expect.objectContaining({
path: '/compose/plex/docker-compose.yaml',
discoveredFrom: 'docker-ls'
})
);
});
it('should scan filesystem if not found in docker ls', async () => {
mockCache.getProject.mockResolvedValue(undefined);
mockCache.load.mockResolvedValue({
lastScan: new Date().toISOString(),
searchPaths: ['/compose'],
projects: {}
});
mockProjectLister.listComposeProjects.mockResolvedValue([]);
mockScanner.findComposeFiles.mockResolvedValue([
'/compose/plex/docker-compose.yaml'
]);
mockScanner.extractProjectName.mockReturnValue('plex');
mockScanner.parseComposeName.mockResolvedValue(null);
const host = { name: 'test', host: 'localhost' };
const result = await discovery.resolveProjectPath(host, 'plex');
expect(result).toBe('/compose/plex/docker-compose.yaml');
expect(mockScanner.findComposeFiles).toHaveBeenCalled();
});
it('should throw error if project not found', async () => {
mockCache.getProject.mockResolvedValue(undefined);
mockCache.load.mockResolvedValue({
lastScan: new Date().toISOString(),
searchPaths: ['/compose'],
projects: {}
});
mockProjectLister.listComposeProjects.mockResolvedValue([]);
mockScanner.findComposeFiles.mockResolvedValue([]);
const host = { name: 'test', host: 'localhost' };
await expect(discovery.resolveProjectPath(host, 'missing')).rejects.toThrow(
"Project 'missing' not found on host 'test'"
);
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/services/compose-discovery.test.ts`
Expected: FAIL with "Cannot find module './compose-discovery.js'"
**Step 3a: Implement ComposeDiscovery - Basic constructor and helper**
```typescript
// src/services/compose-discovery.ts
import type { HostConfig } from '../types.js';
import type { IComposeProjectLister, ISSHService } from './interfaces.js';
import type { ComposeProjectCache, CachedProject } from './compose-cache.js';
import type { ComposeScanner } from './compose-scanner.js';
const DEFAULT_SEARCH_PATHS = ['/compose', '/mnt/cache/compose', '/mnt/cache/code'];
export class ComposeDiscovery {
constructor(
private projectLister: IComposeProjectLister, // Use interface, not IComposeService
private cache: ComposeProjectCache,
private scanner: ComposeScanner,
private sshService: ISSHService
) {}
private getSearchPaths(host: HostConfig, cachedPaths: string[]): string[] {
const paths = new Set<string>();
// Add default paths
DEFAULT_SEARCH_PATHS.forEach(p => paths.add(p));
// Add cached paths
cachedPaths.forEach(p => paths.add(p));
// Add user-configured paths
if (host.composeSearchPaths) {
host.composeSearchPaths.forEach(p => paths.add(p));
}
return Array.from(paths);
}
}
```
**Step 3a-verify: Run test to verify constructor works**
Run: `pnpm test src/services/compose-discovery.test.ts`
Expected: Tests should compile (constructor created) but fail at runtime (missing methods)
**Step 3b: Implement discoverFromDockerLs method**
```typescript
// src/services/compose-discovery.ts - add to class
import { logError } from '../utils/errors.js';
export class ComposeDiscovery {
// ... existing code ...
private async discoverFromDockerLs(
host: HostConfig,
projectName: string
): Promise<CachedProject | null> {
try {
const projects = await this.projectLister.listComposeProjects(host);
const found = projects.find(p => p.name === projectName);
if (found && found.configFiles.length > 0) {
return {
path: found.configFiles[0],
name: projectName,
discoveredFrom: 'docker-ls',
lastSeen: new Date().toISOString()
};
}
} catch (error) {
logError(error as Error, {
operation: 'discoverFromDockerLs',
metadata: { host: host.name, project: projectName }
});
}
return null;
}
}
```
**Step 3b-verify: Run test to verify discoverFromDockerLs passes**
Run: `pnpm test src/services/compose-discovery.test.ts -t "docker compose ls"`
Expected: PASS (tests that call discoverFromDockerLs indirectly should now pass)
**Step 3c: Implement discoverFromFilesystem method**
```typescript
// src/services/compose-discovery.ts - add to class
export class ComposeDiscovery {
// ... existing code ...
private async discoverFromFilesystem(
host: HostConfig,
projectName: string
): Promise<CachedProject | null> {
try {
const cacheData = await this.cache.load(host.name);
const searchPaths = this.getSearchPaths(host, cacheData.searchPaths);
const files = await this.scanner.findComposeFiles(host, searchPaths);
// Parse all files in parallel
const projects = await Promise.all(
files.map(async (file) => {
const dirName = this.scanner.extractProjectName(file);
const explicitName = await this.scanner.parseComposeName(host, file);
const name = explicitName ?? dirName;
return { name, path: file };
})
);
const found = projects.find(p => p.name === projectName);
if (found) {
return {
path: found.path,
name: found.name,
discoveredFrom: 'scan',
lastSeen: new Date().toISOString()
};
}
} catch (error) {
logError(error as Error, {
operation: 'discoverFromFilesystem',
metadata: { host: host.name, project: projectName }
});
}
return null;
}
}
```
**Step 3c-verify: Run test to verify discoverFromFilesystem passes**
Run: `pnpm test src/services/compose-discovery.test.ts -t "scan filesystem"`
Expected: PASS (tests that scan filesystem should now pass)
**Step 3d: Implement resolveProjectPath orchestration method**
```typescript
// src/services/compose-discovery.ts - add to class
export class ComposeDiscovery {
// ... existing code ...
/**
* Resolve compose file path for a project
* Strategy:
* 1. Check cache (trust it - lazy invalidation at handler level)
* 2. Check docker compose ls (running projects) - FAST
* 3. Scan filesystem if not found - SLOWER
* 4. Error if not found
*/
async resolveProjectPath(host: HostConfig, projectName: string): Promise<string> {
// Step 1: Check cache (no validation - lazy invalidation)
const cached = await this.cache.getProject(host.name, projectName);
if (cached) {
return cached.path;
}
// Step 2: Try docker ls first (fast, authoritative for running projects)
const dockerLsResult = await this.discoverFromDockerLs(host, projectName);
if (dockerLsResult) {
await this.cache.updateProject(host.name, projectName, dockerLsResult);
return dockerLsResult.path;
}
// Step 3: Fallback to filesystem scan (slower, but finds stopped projects)
const scanResult = await this.discoverFromFilesystem(host, projectName);
if (scanResult) {
await this.cache.updateProject(host.name, projectName, scanResult);
return scanResult.path;
}
// Step 4: Not found
const cacheData = await this.cache.load(host.name);
const searchPaths = cacheData.searchPaths.length > 0
? cacheData.searchPaths
: DEFAULT_SEARCH_PATHS;
throw new Error(
`Project '${projectName}' not found on host '${host.name}'\n` +
`Searched locations: ${searchPaths.join(', ')}\n` +
`Tip: Provide compose_file parameter if project is in a different location`
);
}
}
```
**Step 4: Run all tests to verify full integration**
Run: `pnpm test src/services/compose-discovery.test.ts`
Expected: ALL PASS
**Step 5: Commit**
```bash
git add src/services/compose-discovery.ts src/services/compose-discovery.test.ts
git commit -m "feat: implement compose project discovery with multi-layer resolution"
```
---
## Task 5: Add Lazy Cache Invalidation (REMOVED - Implemented in Task 8)
**Decision**: After plan validation, we determined that **lazy invalidation should happen at the handler level**, not in ComposeDiscovery.
**Rationale:**
1. Proactive `fileExists()` checks defeat caching purpose (requires remote call on every cache hit)
2. Cache invalidation should be lazy: only invalidate when actual compose operation fails
3. ComposeDiscovery should trust the cache and return paths quickly
4. Handler layer (compose tool handlers) will catch operation failures and trigger cache invalidation
**✅ Implementation in Task 8 Step 4** where ALL compose handlers wrap operations in try-catch blocks that:
- Catch file-not-found errors
- Call `services.composeDiscovery.cache.removeProject(host.name, projectName)`
- Throw helpful error suggesting retry or manual path specification
**Original Task 5 is now a NO-OP** - ComposeDiscovery.resolveProjectPath() returns cached paths directly without validation checks.
---
## Task 6: Make Host Parameter Optional in Compose Schemas
**Files:**
- Modify: `src/schemas/flux/compose.ts`
- Test: `src/schemas/flux/compose.test.ts` (existing file)
**Step 1: Write the failing test**
Add new tests to `src/schemas/flux/compose.test.ts`:
```typescript
// src/schemas/flux/compose.test.ts - ADD these tests to the end of composeListSchema describe block
it('should accept compose:list without host parameter', () => {
const input = {
action: 'compose',
subaction: 'list'
};
const result = composeListSchema.parse(input);
expect(result.host).toBeUndefined();
});
// ADD these tests to the end of composeStatusSchema describe block
it('should accept compose:status without host parameter', () => {
const input = {
action: 'compose',
subaction: 'status',
project: 'plex'
};
const result = composeStatusSchema.parse(input);
expect(result.host).toBeUndefined();
});
// ADD these tests to the end of composeUpSchema describe block
it('should accept compose:up without host parameter', () => {
const input = {
action: 'compose',
subaction: 'up',
project: 'plex'
};
const result = composeUpSchema.parse(input);
expect(result.host).toBeUndefined();
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/schemas/flux/compose.test.ts`
Expected: FAIL with validation error "host is required"
**Step 3: Make host optional in all compose schemas**
```typescript
// src/schemas/flux/compose.ts - modify each schema
export const composeListSchema = z.preprocess(
preprocessWithDiscriminator,
z.object({
action_subaction: z.literal("compose:list"),
action: z.literal("compose"),
subaction: z.literal("list"),
host: hostSchema.optional(), // Changed from required to optional
name_filter: z.string().optional().describe("Partial match on project name"),
...paginationSchema.shape,
response_format: responseFormatSchema
})
.describe("List all Docker Compose projects")
);
export const composeStatusSchema = z.preprocess(
preprocessWithDiscriminator,
z.object({
action_subaction: z.literal("compose:status"),
action: z.literal("compose"),
subaction: z.literal("status"),
host: hostSchema.optional(), // Changed
project: projectSchema,
service_filter: z.string().optional().describe("Filter to specific service(s)"),
...paginationSchema.shape,
response_format: responseFormatSchema
})
.describe("Get Docker Compose project status")
);
// Repeat for all remaining compose schemas:
// - composeUpSchema
// - composeDownSchema
// - composeRestartSchema
// - composeLogsSchema
// - composeBuildSchema
// - composePullSchema
// - composeRecreateSchema
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/schemas/flux/compose.test.ts`
Expected: PASS
**Step 5: Remove obsolete tests that verified host was required**
Now that host is optional, remove tests that verified it was required:
```typescript
// src/schemas/flux/compose.test.ts - REMOVE this test
it("should require host", () => {
expect(() =>
composeListSchema.parse({
action: "compose",
subaction: "list"
})
).toThrow();
});
// CHANGE this test from "should require host and project" to "should require project":
it("should require project", () => {
expect(() =>
composeStatusSchema.parse({
action: "compose",
subaction: "status"
})
).toThrow();
});
```
**Step 6: Commit**
```bash
git add src/schemas/flux/compose.ts src/schemas/flux/compose.test.ts
git commit -m "feat: make host parameter optional in compose schemas"
```
---
## Task 7: Implement Auto-Host Resolution for Compose Operations
**Files:**
- Create: `src/services/host-resolver.ts`
- Test: `src/services/host-resolver.test.ts`
**Step 1: Write the failing test**
```typescript
// src/services/host-resolver.test.ts
import { describe, it, expect, vi } from 'vitest';
import { HostResolver } from './host-resolver.js';
import type { ComposeDiscovery } from './compose-discovery.js';
describe('HostResolver', () => {
const mockDiscovery = {
resolveProjectPath: vi.fn()
} as any;
const resolver = new HostResolver(mockDiscovery);
const hosts = [
{ name: 'tootie', host: '192.168.1.1' },
{ name: 'squirts', host: '192.168.1.2' },
{ name: 'code-server', host: 'localhost' }
];
it('should return specified host if provided', async () => {
const result = await resolver.resolveHost(hosts, 'tootie', 'plex');
expect(result.name).toBe('tootie');
});
it('should auto-resolve to single matching host', async () => {
mockDiscovery.resolveProjectPath
.mockRejectedValueOnce(new Error('Not found')) // tootie
.mockResolvedValueOnce('/compose/plex/docker-compose.yaml') // squirts
.mockRejectedValueOnce(new Error('Not found')); // code-server
const result = await resolver.resolveHost(hosts, undefined, 'plex');
expect(result.name).toBe('squirts');
});
it('should throw error if project found on multiple hosts', async () => {
mockDiscovery.resolveProjectPath
.mockResolvedValueOnce('/compose/plex/docker-compose.yaml') // tootie
.mockResolvedValueOnce('/mnt/cache/compose/plex/docker-compose.yaml') // squirts
.mockRejectedValueOnce(new Error('Not found')); // code-server
await expect(resolver.resolveHost(hosts, undefined, 'plex')).rejects.toThrow(
"Project 'plex' exists on multiple hosts: tootie, squirts. Please specify host parameter."
);
});
it('should throw error if project not found on any host', async () => {
mockDiscovery.resolveProjectPath.mockRejectedValue(new Error('Not found'));
await expect(resolver.resolveHost(hosts, undefined, 'missing')).rejects.toThrow(
"Project 'missing' not found on any configured host"
);
});
it('should throw error if no hosts configured', async () => {
await expect(resolver.resolveHost([], undefined, 'plex')).rejects.toThrow(
'No hosts configured'
);
});
it('should throw error if specified host not found', async () => {
await expect(resolver.resolveHost(hosts, 'invalid', 'plex')).rejects.toThrow(
"Host 'invalid' not found in configuration"
);
});
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/services/host-resolver.test.ts`
Expected: FAIL with "Cannot find module './host-resolver.js'"
**Step 3: Implement HostResolver**
```typescript
// src/services/host-resolver.ts
import type { HostConfig } from '../types.js';
import type { ComposeDiscovery } from './compose-discovery.js';
export class HostResolver {
constructor(private discovery: ComposeDiscovery) {}
/**
* Resolve which host to use for a compose operation
* If host is specified, validate and return it
* If host is not specified, auto-discover from all hosts
*/
async resolveHost(
hosts: HostConfig[],
specifiedHost: string | undefined,
projectName: string
): Promise<HostConfig> {
if (hosts.length === 0) {
throw new Error('No hosts configured');
}
// If host specified, validate and return
if (specifiedHost) {
const host = hosts.find(h => h.name === specifiedHost);
if (!host) {
throw new Error(`Host '${specifiedHost}' not found in configuration`);
}
return host;
}
// Auto-discover: check all hosts in parallel
const results = await Promise.allSettled(
hosts.map(async (host) => {
const path = await this.discovery.resolveProjectPath(host, projectName);
return { host, path };
})
);
const found = results
.filter((r): r is PromiseFulfilledResult<{ host: HostConfig; path: string }> =>
r.status === 'fulfilled'
)
.map(r => r.value);
if (found.length === 0) {
throw new Error(
`Project '${projectName}' not found on any configured host`
);
}
if (found.length > 1) {
const hostNames = found.map(f => f.host.name).join(', ');
throw new Error(
`Project '${projectName}' exists on multiple hosts: ${hostNames}. ` +
`Please specify host parameter.`
);
}
return found[0].host;
}
}
```
**Step 4: Run test to verify it passes**
Run: `pnpm test src/services/host-resolver.test.ts`
Expected: PASS
**Step 5: Commit**
```bash
git add src/services/host-resolver.ts src/services/host-resolver.test.ts
git commit -m "feat: implement auto-host resolution for compose operations"
```
---
## Task 8a: Integrate Discovery into ComposeService
**Files:**
- Modify: `src/services/compose.ts` (add discovery integration)
- Test: `src/services/compose.test.ts` (update existing tests)
**Step 1: Write failing test for ComposeService with discovery integration**
```typescript
// src/services/compose.test.ts - add test
it('should use discovery to resolve compose file path', async () => {
const mockDiscovery = {
resolveProjectPath: vi.fn().mockResolvedValue('/compose/plex/docker-compose.yaml')
};
// Inject discovery into service (via setter or constructor)
composeService.setDiscovery(mockDiscovery as any);
const host = { name: 'test', host: 'localhost' };
await composeService.composeUp(host, 'plex', true);
// Verify discovery was called
expect(mockDiscovery.resolveProjectPath).toHaveBeenCalledWith(host, 'plex');
// Verify docker compose command includes -f flag with discovered path
expect(mockLocalExecutor.executeLocalCommand).toHaveBeenCalledWith(
'docker',
expect.arrayContaining(['-f', '/compose/plex/docker-compose.yaml']),
expect.any(Object)
);
});
it('should fall back gracefully when discovery fails', async () => {
const mockDiscovery = {
resolveProjectPath: vi.fn().mockRejectedValue(new Error('Project not found'))
};
composeService.setDiscovery(mockDiscovery as any);
const host = { name: 'test', host: 'localhost' };
// Should NOT throw - should fall back to project name only
await composeService.composeUp(host, 'plex', true);
// Should NOT include -f flag when discovery fails
expect(mockLocalExecutor.executeLocalCommand).toHaveBeenCalledWith(
'docker',
expect.not.arrayContaining(['-f']),
expect.any(Object)
);
});
```
**Step 2: Run test to verify RED**
Run: `pnpm test src/services/compose.test.ts -t "discovery"`
Expected: FAIL - discovery integration not yet implemented
**Step 3: Update ComposeService to use discovery**
```typescript
// src/services/compose.ts - add discovery property and setter
export class ComposeService implements IComposeService {
private discovery?: ComposeDiscovery;
constructor(
private sshService: ISSHService,
private localExecutor: ILocalExecutorService
) {}
/**
* Set discovery service (called after both services are instantiated)
*/
setDiscovery(discovery: ComposeDiscovery): void {
this.discovery = discovery;
}
// ... existing methods ...
}
// src/services/compose.ts - modify composeExec helper
private async composeExec(
host: HostConfig,
project: string,
action: string,
args: string[]
): Promise<string> {
validateProjectName(project);
validateComposeArgs(args);
// Resolve compose file path via discovery
let composePath: string | undefined;
if (this.discovery) {
try {
composePath = await this.discovery.resolveProjectPath(host, project);
} catch (error) {
// If discovery fails, fall back to project name only
logError(error as Error, {
operation: 'composeExec',
metadata: { host: host.name, project, action }
});
}
}
// Build command with -f flag if path was discovered
const cmdParts = ['docker', 'compose'];
if (composePath) {
cmdParts.push('-f', composePath);
}
cmdParts.push('-p', project, action, ...args);
const command = cmdParts.join(' ');
try {
let stdout: string;
if (isLocalHost(host)) {
stdout = await this.localExecutor.executeLocalCommand(
'docker',
['compose', ...(composePath ? ['-f', composePath] : []), '-p', project, action, ...args],
{ timeoutMs: 120000 }
);
} else {
validateHostForSsh(host);
stdout = await this.sshService.executeSSHCommand(host, command, [], {
timeoutMs: 120000
});
}
return stdout;
} catch (error) {
// ... existing error handling
}
}
```
**Step 4: Run test to verify GREEN**
Run: `pnpm test src/services/compose.test.ts -t "discovery"`
Expected: ALL PASS - discovery integration working
**Step 5: Update ComposeDiscovery to use interface instead of concrete class**
```typescript
// src/services/compose-discovery.ts - update constructor
export class ComposeDiscovery {
constructor(
private projectLister: IComposeProjectLister, // Changed from IComposeService
private cache: ComposeProjectCache,
private scanner: ComposeScanner,
private sshService: ISSHService
) {}
private async discoverFromDockerLs(
host: HostConfig,
projectName: string
): Promise<CachedProject | null> {
try {
// Use projectLister interface instead of composeService
const projects = await this.projectLister.listComposeProjects(host);
const found = projects.find(p => p.name === projectName);
// ... rest of method
}
}
// Update all other methods that used composeService to use projectLister
}
```
**Step 6: ComposeService implements IComposeProjectLister (no changes needed)**
ComposeService already has `listComposeProjects()` method, so it implicitly implements the interface via TypeScript's structural typing.
> **⚠️ Architectural Note:** This implementation creates a bidirectional dependency (ComposeService ↔ ComposeDiscovery via setter injection). This is a known coupling pattern that works but could be improved in future iterations using an event-based pattern or by making discovery fully standalone. For the current implementation, this pattern is acceptable as it's isolated within the service container.
**Step 7: Create cache invalidation utility (DRY extraction)**
```typescript
// src/tools/handlers/compose-utils.ts (new file)
import type { ComposeDiscovery } from '../../services/compose-discovery.js';
import { logError } from '../../utils/errors.js';
/**
* Check if error is a file-not-found error
*/
function isFileNotFoundError(error: unknown): boolean {
return error instanceof Error &&
(error.message.includes('No such file') ||
error.message.includes('not found') ||
error.message.includes('Cannot find'));
}
/**
* Wrapper for compose operations with automatic cache invalidation
* on file-not-found errors (lazy invalidation pattern)
*/
export async function withCacheInvalidation<T>(
operation: () => Promise<T>,
projectName: string,
hostName: string,
discovery: ComposeDiscovery,
operationName: string
): Promise<T> {
try {
return await operation();
} catch (error) {
if (isFileNotFoundError(error)) {
// Invalidate cached path
await discovery.cache.removeProject(hostName, projectName);
logError(error as Error, {
operation: operationName,
metadata: { host: hostName, project: projectName, cacheInvalidated: true }
});
throw new Error(
`Compose file not found for project '${projectName}' on host '${hostName}'.\n` +
`Cache has been invalidated. Please retry the operation or use the compose_file parameter to specify the path explicitly.`
);
}
// Re-throw other errors without invalidation
throw error;
}
}
```
**Step 8: Initialize discovery in ServiceContainer**
```typescript
// src/services/container.ts - modify getComposeService method
import { ComposeProjectCache } from './compose-cache.js';
import { ComposeScanner } from './compose-scanner.js';
import { ComposeDiscovery } from './compose-discovery.js';
class ServiceContainer {
private composeDiscovery?: ComposeDiscovery;
getComposeService(): IComposeService {
if (!this.composeService) {
this.composeService = new ComposeService(
this.getSSHService(),
this.getLocalExecutor()
);
// Initialize discovery after compose service is created
const cache = new ComposeProjectCache();
const scanner = new ComposeScanner(this.getSSHService(), this.getLocalExecutor());
this.composeDiscovery = new ComposeDiscovery(
this.composeService, // Injected as IComposeProjectLister interface
cache,
scanner,
this.getSSHService()
);
// Inject discovery back into compose service (bidirectional dependency)
this.composeService.setDiscovery(this.composeDiscovery);
}
return this.composeService;
}
getComposeDiscovery(): ComposeDiscovery {
if (!this.composeDiscovery) {
// Ensure compose service is initialized (which creates discovery)
this.getComposeService();
}
return this.composeDiscovery!;
}
}
```
**Step 9: Update compose handlers using DRY utility**
```typescript
// src/tools/handlers/compose.ts - example for composeUp
import { HostResolver } from '../../services/host-resolver.js';
import { withCacheInvalidation } from './compose-utils.js';
export async function handleComposeUp(
input: ComposeUpInput,
hosts: HostConfig[],
services: Services
): Promise<string> {
// Resolve host (may auto-resolve if host param omitted)
const resolver = new HostResolver(services.composeDiscovery);
const host = await resolver.resolveHost(hosts, input.host, input.project);
// Execute operation with automatic cache invalidation
return withCacheInvalidation(
async () => {
const result = await services.composeService.composeUp(host, input.project, input.detach);
return formatComposeResult('up', host.name, input.project, result);
},
input.project,
host.name,
services.composeDiscovery,
'handleComposeUp'
);
}
// Apply same pattern to ALL compose handlers (handleComposeDown, handleComposeRestart, etc.)
// Each handler should wrap the compose operation with withCacheInvalidation()
```
**Step 10: Update service interfaces**
```typescript
// src/services/interfaces.ts - add ComposeDiscovery to Services
export interface Services {
dockerService: IDockerService;
composeService: IComposeService;
containerService: IContainerService;
composeDiscovery: ComposeDiscovery; // Add this
// ... other services
}
```
**Step 11: Commit**
```bash
git add src/services/compose.ts src/services/compose.test.ts src/tools/handlers/compose.ts src/services/interfaces.ts src/index.ts
git commit -m "feat: integrate compose discovery into compose operations with TDD"
```
---
## Task 9: Add compose:refresh Subaction
**Files:**
- Modify: `src/schemas/flux/compose.ts`
- Create handler in `src/tools/handlers/compose.ts`
- Test: `src/schemas/flux/compose.test.ts`
**Step 1: Write the failing test**
```typescript
// src/schemas/flux/compose.test.ts - add test
it('should accept compose:refresh action', () => {
const input = {
action_subaction: 'compose:refresh',
action: 'compose',
subaction: 'refresh',
host: 'tootie'
};
const result = composeRefreshSchema.parse(input);
expect(result.action).toBe('compose');
expect(result.subaction).toBe('refresh');
});
it('should accept compose:refresh without host (all hosts)', () => {
const input = {
action_subaction: 'compose:refresh',
action: 'compose',
subaction: 'refresh'
};
const result = composeRefreshSchema.parse(input);
expect(result.host).toBeUndefined();
});
```
**Step 2: Run test to verify it fails**
Run: `pnpm test src/schemas/flux/compose.test.ts -t "refresh"`
Expected: FAIL with "composeRefreshSchema is not defined"
**Step 3: Add refresh schema**
```typescript
// src/schemas/flux/compose.ts
export const composeRefreshSchema = z.preprocess(
preprocessWithDiscriminator,
z
.object({
action_subaction: z.literal("compose:refresh"),
action: z.literal("compose"),
subaction: z.literal("refresh"),
host: hostSchema.optional().describe("Specific host to refresh, or all hosts if omitted"),
response_format: responseFormatSchema
})
.describe("Refresh compose project cache (force rescan)")
);
export type ComposeRefreshInput = z.infer<typeof composeRefreshSchema>;
```
**Step 4: Add to exports and union type**
```typescript
// src/schemas/flux/compose.ts - update union type
export type ComposeActionInput =
| ComposeListInput
| ComposeStatusInput
| ComposeUpInput
| ComposeDownInput
| ComposeRestartInput
| ComposeLogsInput
| ComposeBuildInput
| ComposePullInput
| ComposeRecreateInput
| ComposeRefreshInput; // Add this
```
**Step 5a: Write tests for discovery helper methods**
```typescript
// src/services/compose-discovery.test.ts - add tests
describe('ComposeDiscovery - refresh cache helpers', () => {
it('should discover all projects from docker ls', async () => {
mockProjectLister.listComposeProjects.mockResolvedValue([
{ name: 'plex', status: 'running', configFiles: ['/compose/plex/docker-compose.yaml'], services: [] },
{ name: 'sonarr', status: 'running', configFiles: ['/compose/sonarr/docker-compose.yaml'], services: [] }
]);
// Call private method via public refreshCache
const host = { name: 'test', host: 'localhost' };
await discovery.refreshCache(host);
// Verify cache was updated with both projects
expect(mockCache.save).toHaveBeenCalledWith('test', expect.objectContaining({
projects: expect.objectContaining({
plex: expect.objectContaining({ path: '/compose/plex/docker-compose.yaml' }),
sonarr: expect.objectContaining({ path: '/compose/sonarr/docker-compose.yaml' })
})
}));
});
it('should merge docker-ls and filesystem results with docker-ls taking precedence', async () => {
mockProjectLister.listComposeProjects.mockResolvedValue([
{ name: 'plex', status: 'running', configFiles: ['/compose/plex/docker-compose.yaml'], services: [] }
]);
mockCache.load.mockResolvedValue({ lastScan: '', searchPaths: [], projects: {} });
mockScanner.findComposeFiles.mockResolvedValue(['/compose/radarr/docker-compose.yaml']);
mockScanner.extractProjectName.mockReturnValue('radarr');
mockScanner.parseComposeName.mockResolvedValue(null);
const host = { name: 'test', host: 'localhost' };
await discovery.refreshCache(host);
// Should have both projects
expect(mockCache.save).toHaveBeenCalledWith('test', expect.objectContaining({
projects: expect.objectContaining({
plex: expect.any(Object),
radarr: expect.any(Object)
})
}));
});
});
```
**Step 5b: Run test to verify RED**
Run: `pnpm test src/services/compose-discovery.test.ts -t "refresh cache"`
Expected: FAIL - refreshCache method not yet implemented
**Step 5c: Add refresh method to ComposeDiscovery**
```typescript
// src/services/compose-discovery.ts
/**
* Force refresh project cache for a host
* Discovers all projects and updates cache in one operation
*/
async refreshCache(host: HostConfig): Promise<void> {
const cacheData = await this.cache.load(host.name);
const searchPaths = this.getSearchPaths(host, cacheData.searchPaths);
// Get all projects from both sources
const dockerProjects = await this.discoverAllFromDockerLs(host);
const filesystemProjects = await this.discoverAllFromFilesystem(host, searchPaths);
// Merge results (docker-ls takes precedence)
const projects: Record<string, CachedProject> = {};
for (const project of dockerProjects) {
projects[project.name] = project;
}
for (const project of filesystemProjects) {
// Don't overwrite docker-ls entries (they're more authoritative)
if (!projects[project.name]) {
projects[project.name] = project;
}
}
// Update cache
await this.cache.save(host.name, {
lastScan: new Date().toISOString(),
searchPaths,
projects
});
}
/**
* Discover ALL projects from docker ls (not just one by name)
* Reuses discoverFromDockerLs logic
*/
private async discoverAllFromDockerLs(host: HostConfig): Promise<CachedProject[]> {
try {
const projects = await this.projectLister.listComposeProjects(host);
return projects
.filter(p => p.configFiles.length > 0)
.map(p => ({
path: p.configFiles[0],
name: p.name,
discoveredFrom: 'docker-ls' as const,
lastSeen: new Date().toISOString()
}));
} catch (error) {
logError(error as Error, {
operation: 'discoverAllFromDockerLs',
metadata: { host: host.name }
});
return [];
}
}
/**
* Discover ALL projects from filesystem scan
* Reuses discoverFromFilesystem logic
*/
private async discoverAllFromFilesystem(
host: HostConfig,
searchPaths: string[]
): Promise<CachedProject[]> {
try {
const files = await this.scanner.findComposeFiles(host, searchPaths);
const projects = await Promise.all(
files.map(async (file) => {
const dirName = this.scanner.extractProjectName(file);
const explicitName = await this.scanner.parseComposeName(host, file);
const name = explicitName ?? dirName;
return {
path: file,
name,
discoveredFrom: 'scan' as const,
lastSeen: new Date().toISOString()
};
})
);
return projects;
} catch (error) {
logError(error as Error, {
operation: 'discoverAllFromFilesystem',
metadata: { host: host.name, searchPaths }
});
return [];
}
}
```
**Note**: The `discoverFromDockerLs()` and `discoverFromFilesystem()` methods from Task 4 find a *single* project by name. These new methods `discoverAllFromDockerLs()` and `discoverAllFromFilesystem()` discover *all* projects, which is needed for cache refresh. The logic is similar but the use case is different (single vs all), so they're separate methods with clear names.
**Step 6: Add handler**
```typescript
// src/tools/handlers/compose.ts
export async function handleComposeRefresh(
input: ComposeRefreshInput,
hosts: HostConfig[],
services: Services
): Promise<string> {
const hostsToRefresh = input.host
? hosts.filter(h => h.name === input.host)
: hosts;
if (hostsToRefresh.length === 0) {
throw new Error(
input.host
? `Host '${input.host}' not found in configuration`
: 'No hosts configured'
);
}
// Refresh all hosts in parallel
await Promise.all(
hostsToRefresh.map(host => services.composeDiscovery.refreshCache(host))
);
const hostNames = hostsToRefresh.map(h => h.name).join(', ');
return `✓ Refreshed compose project cache for: ${hostNames}`;
}
```
**Step 7: Run test to verify it passes**
Run: `pnpm test src/schemas/flux/compose.test.ts -t "refresh"`
Expected: PASS
**Step 8: Update schema count in flux index**
```typescript
// src/schemas/flux/index.ts - update comment
/**
* Flux Tool Schema - Docker infrastructure management
*
* Actions: 4 (container, compose, docker, host)
* Subactions: 40 total // Changed from 39
* - container: 14
* - compose: 10 (added refresh) // Changed from 9
* - docker: 9
* - host: 7
*/
```
**Step 9: Commit**
```bash
git add src/schemas/flux/compose.ts src/schemas/flux/index.ts src/services/compose-discovery.ts src/tools/handlers/compose.ts src/schemas/flux/compose.test.ts
git commit -m "feat: add compose:refresh subaction for manual cache refresh"
```
---
## Task 10: Add .cache/ to .gitignore
**Files:**
- Modify: `.gitignore`
**Step 1: Add cache directory to .gitignore**
```bash
echo "" >> .gitignore
echo "# Compose project cache" >> .gitignore
echo ".cache/compose-projects/" >> .gitignore
```
**Step 2: Verify gitignore**
Run: `git status`
Expected: `.cache/compose-projects/` should not appear in untracked files
**Step 3: Commit**
```bash
git add .gitignore
git commit -m "chore: add compose project cache to gitignore"
```
---
## Task 11: Update Documentation
**Files:**
- Modify: `README.md`
- Create: `docs/compose-discovery.md`
**Step 1: Create detailed discovery documentation**
```markdown
<!-- docs/compose-discovery.md -->
# Docker Compose Auto-Discovery
## Overview
The synapse MCP automatically discovers and caches Docker Compose project locations, eliminating the need to specify file paths manually.
## How It Works
### Discovery Layers
1. **Cache Check**: Fast lookup in local cache (`.cache/compose-projects/{hostname}.json`)
2. **Running Projects**: Query `docker compose ls` for active stacks
3. **Filesystem Scan**: Search configured directories for compose files (depth 3)
4. **Cache & Return**: Store discovered path for future use
### Search Paths
**Default locations:**
- `/compose`
- `/mnt/cache/compose`
- `/mnt/cache/code`
**Custom paths** (optional in `synapse.config.json`):
```json
{
"hosts": [
{
"name": "myhost",
"host": "192.168.1.100",
"composeSearchPaths": ["/opt/stacks", "/srv/docker"]
}
]
}
```
**Auto-learned paths:**
- Parent directories from running projects
- Parent directories from user-provided `compose_file` paths
### Project Name Resolution
Projects are identified by:
1. **Explicit `name:` field** in compose file (preferred)
2. **Parent directory name** (fallback)
Example:
```yaml
# /mnt/cache/compose/my-app/docker-compose.yaml
name: custom-name # Uses "custom-name"
# OR (no name field)
services:
web:
image: nginx # Uses "my-app" (directory name)
```
## Usage
### Automatic Host Resolution
If a project exists on only one host, you can omit the `host` parameter:
```typescript
// Project exists only on "tootie"
flux({ action: "compose", subaction: "up", project: "plex" })
// Auto-resolves to tootie
// Project exists on multiple hosts
flux({ action: "compose", subaction: "up", project: "postgres" })
// Error: "Project exists on multiple hosts: tootie, squirts. Specify host parameter."
```
### Manual Cache Refresh
Force rescan of compose files:
```typescript
// Refresh all hosts
flux({ action: "compose", subaction: "refresh" })
// Refresh specific host
flux({ action: "compose", subaction: "refresh", host: "tootie" })
```
### User-Provided Paths
If a project isn't found automatically, you can specify the path:
```typescript
flux({
action: "compose",
subaction: "up",
project: "myapp",
compose_file: "/custom/location/docker-compose.yaml"
})
// Path will be cached for future use
```
## Performance
- **Cache hit**: ~5ms
- **Running project**: ~50ms
- **Filesystem scan**: ~130ms (first time)
- **Multi-host scan**: ~50ms (parallel)
## Cache Invalidation
Cache automatically invalidates when:
- File doesn't exist at cached path (triggers rescan)
- Manual refresh requested
- 5+ minutes since last scan (for running projects only)
## Troubleshooting
### Project Not Found
```
Error: Project 'myapp' not found on host 'tootie'
Searched locations: /compose, /mnt/cache/compose, /mnt/cache/code
```
**Solutions:**
1. Verify compose file exists in a searched location
2. Add custom search path to config
3. Use `compose_file` parameter
4. Run `compose:refresh` to force rescan
### Multiple Projects Found
```
Error: Project 'postgres' exists on multiple hosts: tootie, squirts
```
**Solution:** Specify `host` parameter explicitly
### Stale Cache
Run refresh to clear stale cache:
```typescript
flux({ action: "compose", subaction: "refresh", host: "tootie" })
```
```
**Step 2: Update README.md**
```markdown
<!-- README.md - add to features section -->
## Features
- **Multi-host Docker management** - Control containers, compose stacks, and Docker itself across multiple servers
- **SSH-based remote operations** - Secure command execution on remote hosts
- **Automatic compose project discovery** - No need to specify file paths, automatically finds and caches compose files
- **Auto-host resolution** - Omit host parameter when project is unique across hosts
- **Smart caching** - Fast lookups with automatic invalidation
<!-- Add to usage examples -->
## Quick Start Examples
### Compose Operations (Auto-Discovery)
```typescript
// Start a stack (auto-discovers location and host)
flux({ action: "compose", subaction: "up", project: "plex" })
// Stop a stack on specific host
flux({ action: "compose", subaction: "down", project: "sonarr", host: "tootie" })
// Refresh cache to discover new projects
flux({ action: "compose", subaction: "refresh" })
```
See [docs/compose-discovery.md](docs/compose-discovery.md) for detailed discovery documentation.
```
**Step 3: Commit**
```bash
git add README.md docs/compose-discovery.md
git commit -m "docs: add compose auto-discovery documentation"
```
---
## Task 12: Integration Testing
**Files:**
- Create: `src/services/compose-discovery.integration.test.ts`
**Step 1: Write integration test**
```typescript
// src/services/compose-discovery.integration.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { mkdir, writeFile, rm } from 'fs/promises';
import { join } from 'path';
import { ComposeProjectCache } from './compose-cache.js';
import { ComposeScanner } from './compose-scanner.js';
import { ComposeDiscovery } from './compose-discovery.js';
import { ComposeService } from './compose.js';
import { SSHService } from './ssh.js';
import { LocalExecutorService } from './local-executor.js';
import { SSHPoolService } from './ssh-pool.js';
describe('Compose Discovery Integration', () => {
const testDir = '/tmp/synapse-discovery-test';
const cacheDir = join(testDir, '.cache');
beforeAll(async () => {
// Create test compose files
await mkdir(join(testDir, 'plex'), { recursive: true });
await mkdir(join(testDir, 'sonarr'), { recursive: true });
await mkdir(join(testDir, 'custom-name'), { recursive: true });
await writeFile(
join(testDir, 'plex/docker-compose.yaml'),
'services:\n plex:\n image: plexinc/pms-docker\n'
);
await writeFile(
join(testDir, 'sonarr/docker-compose.yml'),
'services:\n sonarr:\n image: linuxserver/sonarr\n'
);
await writeFile(
join(testDir, 'custom-name/docker-compose.yaml'),
'name: my-app\nservices:\n web:\n image: nginx\n'
);
});
afterAll(async () => {
await rm(testDir, { recursive: true, force: true });
});
it('should discover projects from filesystem', async () => {
const sshPool = new SSHPoolService();
const localExecutor = new LocalExecutorService();
const sshService = new SSHService(sshPool);
const composeService = new ComposeService(sshService, localExecutor);
const cache = new ComposeProjectCache(cacheDir);
const scanner = new ComposeScanner(sshService, localExecutor);
const discovery = new ComposeDiscovery(composeService, cache, scanner);
const host = { name: 'localhost', host: 'localhost' };
// Override search paths to use test directory
const cacheData = {
lastScan: new Date().toISOString(),
searchPaths: [testDir],
projects: {}
};
await cache.save('localhost', cacheData);
// Discover plex
const plexPath = await discovery.resolveProjectPath(host, 'plex');
expect(plexPath).toBe(join(testDir, 'plex/docker-compose.yaml'));
// Discover sonarr
const sonarrPath = await discovery.resolveProjectPath(host, 'sonarr');
expect(sonarrPath).toBe(join(testDir, 'sonarr/docker-compose.yml'));
// Discover project with explicit name
const customPath = await discovery.resolveProjectPath(host, 'my-app');
expect(customPath).toBe(join(testDir, 'custom-name/docker-compose.yaml'));
// Verify cache was updated
const cached = await cache.getProject('localhost', 'plex');
expect(cached?.path).toBe(join(testDir, 'plex/docker-compose.yaml'));
});
it('should return cached path without validation (lazy invalidation)', async () => {
const sshPool = new SSHPoolService();
const localExecutor = new LocalExecutorService();
const sshService = new SSHService(sshPool);
const composeService = new ComposeService(sshService, localExecutor);
const cache = new ComposeProjectCache(cacheDir);
const scanner = new ComposeScanner(sshService, localExecutor);
const discovery = new ComposeDiscovery(composeService, cache, scanner, sshService);
const host = { name: 'localhost', host: 'localhost' };
// Cache a non-existent path
await cache.updateProject('localhost', 'test', {
path: '/nonexistent/docker-compose.yaml',
name: 'test',
discoveredFrom: 'user-provided',
lastSeen: new Date().toISOString()
});
// Discovery layer trusts cache and returns path without validation
// (Lazy invalidation happens at handler level when operation fails)
const path = await discovery.resolveProjectPath(host, 'test');
expect(path).toBe('/nonexistent/docker-compose.yaml');
// Cache should NOT be invalidated by discovery layer
const cached = await cache.getProject('localhost', 'test');
expect(cached).toBeDefined();
expect(cached?.path).toBe('/nonexistent/docker-compose.yaml');
});
});
```
**Step 2: Run integration test**
Run: `pnpm test src/services/compose-discovery.integration.test.ts`
Expected: PASS
**Step 3: Commit**
```bash
git add src/services/compose-discovery.integration.test.ts
git commit -m "test: add integration tests for compose discovery"
```
---
## Task 13: Final Verification and Cleanup
**Step 1: Run all tests**
Run: `pnpm test`
Expected: ALL PASS
**Step 2: Run type check**
Run: `pnpm run typecheck`
Expected: No errors
**Step 3: Run linter**
Run: `pnpm run lint`
Expected: No errors
**Step 4: Build project**
Run: `pnpm run build`
Expected: Successful build
**Step 5: Manual testing (if available)**
```bash
# Start MCP server
pnpm run dev
# Test compose operations in MCP client
# 1. Try compose:up without host parameter
# 2. Try compose:refresh
# 3. Verify cache files created in .cache/compose-projects/
```
**Step 6: Final commit**
```bash
git add .
git commit -m "feat: complete compose auto-discovery implementation
- Auto-discovers compose files across hosts
- Optional host parameter with auto-resolution
- Persistent cache with smart invalidation
- Manual refresh capability
- Configurable search paths
- Parallel discovery for performance"
```
---
## Implementation Complete
Total tasks: 14 (Task 0-13)
Estimated time: 4-6 hours
**Next steps:**
1. Choose execution approach (subagent-driven or parallel session)
2. Execute plan task-by-task
3. Test with real homelab environment
4. Iterate on any edge cases discovered
**Key features delivered:**
✓ Automatic compose file discovery
✓ Per-host caching with JSON persistence
✓ Optional host parameter
✓ Auto-host resolution for unique projects
✓ Cache invalidation for missing files
✓ Manual refresh capability
✓ Configurable search paths
✓ Parallel execution for performance
✓ Comprehensive test coverage
✓ Full documentation
