custom-mutation-strategies.md•17.3 kB
# Custom Mutation Strategies
## Overview
GEPA's genetic evolution system supports custom mutation strategies that allow developers to implement domain-specific prompt optimization techniques. This guide covers how to create, implement, and integrate custom mutation strategies.
## Architecture
### Base Mutation Strategy Interface
```typescript
interface MutationStrategy {
name: string;
weight: number;
applicable: (candidate: PromptCandidate, context: TaskContext) => boolean;
mutate: (candidate: PromptCandidate, context: TaskContext) => Promise<PromptCandidate[]>;
validate: (original: PromptCandidate, mutated: PromptCandidate) => boolean;
}
```
### Strategy Manager
```typescript
class MutationStrategyManager {
private strategies = new Map<string, MutationStrategy>();
private analytics = new Map<string, StrategyAnalytics>();
registerStrategy(strategy: MutationStrategy): void;
selectStrategy(candidate: PromptCandidate, context: TaskContext): MutationStrategy;
evaluateStrategy(strategyName: string, results: MutationResult[]): void;
}
```
## Built-in Strategies
### 1. Reflective Mutation
Based on trajectory analysis and failure pattern identification.
```typescript
class ReflectiveMutationStrategy implements MutationStrategy {
name = 'reflective';
weight = 0.4;
applicable(candidate: PromptCandidate, context: TaskContext): boolean {
return candidate.rolloutCount > 3 && candidate.averageScore < 0.8;
}
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
// Get recent trajectories
const trajectories = await this.trajectoryStore.query({
promptId: candidate.id,
limit: 5,
failuresOnly: true
});
// Analyze failure patterns
const analysis = await this.reflectionEngine.analyzeBatch(trajectories);
// Generate improvements based on high-confidence suggestions
const mutations: PromptCandidate[] = [];
for (const suggestion of analysis.recommendations) {
if (suggestion.expectedImpact > 0.6) {
const mutatedContent = await this.applyImprovement(
candidate.content,
suggestion
);
const mutation = this.createMutation(candidate, mutatedContent, {
strategy: 'reflective',
sourceAnalysis: suggestion,
confidence: suggestion.expectedImpact
});
mutations.push(mutation);
}
}
return mutations;
}
private async applyImprovement(
content: string,
improvement: PromptImprovement
): Promise<string> {
switch (improvement.type) {
case 'add_instruction':
return this.addInstruction(content, improvement.proposedChange);
case 'clarify_step':
return this.clarifyStep(content, improvement.targetSection, improvement.proposedChange);
case 'add_example':
return this.addExample(content, improvement.proposedChange);
default:
return this.genericImprovement(content, improvement);
}
}
}
```
### 2. Semantic Mutation
Focuses on semantic variations while preserving meaning.
```typescript
class SemanticMutationStrategy implements MutationStrategy {
name = 'semantic';
weight = 0.3;
applicable(candidate: PromptCandidate, context: TaskContext): boolean {
return context.requiredCapabilities.includes('language-understanding');
}
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
const semanticVariations = await this.generateSemanticVariations(
candidate.content,
context
);
return semanticVariations.map(content =>
this.createMutation(candidate, content, {
strategy: 'semantic',
preservedMeaning: true,
variationType: 'paraphrase'
})
);
}
private async generateSemanticVariations(
content: string,
context: TaskContext
): Promise<string[]> {
const variations: string[] = [];
// Synonym replacement
variations.push(await this.synonymReplacement(content));
// Sentence restructuring
variations.push(await this.restructureSentences(content));
// Style adaptation
if (context.category === 'formal') {
variations.push(await this.formalizeLanguage(content));
} else if (context.category === 'conversational') {
variations.push(await this.casualizeLanguage(content));
}
return variations;
}
}
```
### 3. Domain-Specific Mutation
Tailored for specific domains like code generation, creative writing, etc.
```typescript
class CodeGenerationMutationStrategy implements MutationStrategy {
name = 'code-generation';
weight = 0.5;
applicable(candidate: PromptCandidate, context: TaskContext): boolean {
return context.category === 'code-generation' ||
context.requiredCapabilities.includes('programming');
}
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
const mutations: PromptCandidate[] = [];
// Add programming language specificity
const languageSpecific = await this.addLanguageSpecificity(
candidate.content,
context
);
mutations.push(this.createMutation(candidate, languageSpecific, {
strategy: 'code-generation',
enhancement: 'language-specificity'
}));
// Add code quality emphasis
const qualityFocused = await this.emphasizeCodeQuality(candidate.content);
mutations.push(this.createMutation(candidate, qualityFocused, {
strategy: 'code-generation',
enhancement: 'quality-focus'
}));
// Add testing instructions
const testingFocused = await this.addTestingInstructions(candidate.content);
mutations.push(this.createMutation(candidate, testingFocused, {
strategy: 'code-generation',
enhancement: 'testing-focus'
}));
return mutations;
}
private async addLanguageSpecificity(
content: string,
context: TaskContext
): Promise<string> {
// Extract target language from context
const language = this.extractTargetLanguage(context);
if (!language) return content;
const languagePrompt = `
${content}
Additional instructions for ${language}:
- Follow ${language}-specific best practices and conventions
- Use appropriate ${language} idioms and patterns
- Include proper error handling for ${language}
- Consider ${language} performance characteristics
- Use ${language} standard library when appropriate
`;
return languagePrompt;
}
}
```
## Creating Custom Strategies
### Step 1: Define Strategy Class
```typescript
class CustomBusinessLogicStrategy implements MutationStrategy {
name = 'business-logic';
weight = 0.6; // Higher weight for domain-specific tasks
constructor(
private domain: string,
private businessRules: BusinessRule[],
private llmAdapter: LLMAdapter
) {}
applicable(candidate: PromptCandidate, context: TaskContext): boolean {
return context.category === this.domain &&
this.hasBusinessContext(context);
}
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
const mutations: PromptCandidate[] = [];
// Apply each relevant business rule
for (const rule of this.getApplicableRules(context)) {
const enhanced = await this.applyBusinessRule(candidate.content, rule);
mutations.push(this.createMutation(candidate, enhanced, {
strategy: 'business-logic',
appliedRule: rule.id,
ruleDescription: rule.description
}));
}
return mutations;
}
private async applyBusinessRule(
content: string,
rule: BusinessRule
): Promise<string> {
const enhancementPrompt = `
Enhance this prompt to include business rule: ${rule.description}
Original prompt: ${content}
Business rule details:
- Name: ${rule.name}
- Conditions: ${rule.conditions.join(', ')}
- Expected behavior: ${rule.expectedBehavior}
- Validation criteria: ${rule.validationCriteria.join(', ')}
Return an enhanced prompt that incorporates this business logic:
`;
const response = await this.llmAdapter.callLLM(enhancementPrompt);
return response.content;
}
}
```
### Step 2: Define Supporting Types
```typescript
interface BusinessRule {
id: string;
name: string;
description: string;
conditions: string[];
expectedBehavior: string;
validationCriteria: string[];
priority: 'low' | 'medium' | 'high';
}
interface StrategyAnalytics {
applicationsCount: number;
successRate: number;
averageFitnessImprovement: number;
lastUsed: Date;
performanceHistory: PerformanceMetric[];
}
interface MutationResult {
originalFitness: number;
newFitness: number;
fitnessImprovement: number;
strategy: string;
successful: boolean;
metadata: Record<string, any>;
}
```
### Step 3: Register Strategy
```typescript
// In your application setup
const strategyManager = new MutationStrategyManager();
// Register built-in strategies
strategyManager.registerStrategy(new ReflectiveMutationStrategy(dependencies));
strategyManager.registerStrategy(new SemanticMutationStrategy(dependencies));
strategyManager.registerStrategy(new CodeGenerationMutationStrategy(dependencies));
// Register custom strategy
const businessRules: BusinessRule[] = [
{
id: 'customer-satisfaction',
name: 'Customer Satisfaction Priority',
description: 'Always prioritize customer satisfaction in recommendations',
conditions: ['customer-facing', 'recommendation-task'],
expectedBehavior: 'Provide recommendations that maximize customer value',
validationCriteria: ['customer-benefit-mentioned', 'clear-value-proposition'],
priority: 'high'
}
];
strategyManager.registerStrategy(
new CustomBusinessLogicStrategy('customer-service', businessRules, llmAdapter)
);
// Configure in PromptMutator
const promptMutator = new PromptMutator({
strategyManager,
customStrategies: ['business-logic'],
adaptiveWeighting: true
});
```
## Advanced Strategy Features
### Adaptive Weighting
Strategies can adapt their weights based on performance:
```typescript
class AdaptiveStrategyManager extends MutationStrategyManager {
private performanceHistory = new Map<string, number[]>();
updateStrategyWeight(strategyName: string, performance: number): void {
const history = this.performanceHistory.get(strategyName) || [];
history.push(performance);
// Keep only recent history
if (history.length > 20) {
history.shift();
}
this.performanceHistory.set(strategyName, history);
// Adjust weight based on recent performance
const strategy = this.strategies.get(strategyName);
if (strategy) {
const avgPerformance = history.reduce((a, b) => a + b, 0) / history.length;
strategy.weight = Math.max(0.1, Math.min(1.0, avgPerformance));
}
}
}
```
### Multi-Stage Mutation
Complex strategies can involve multiple mutation stages:
```typescript
class MultiStageMutationStrategy implements MutationStrategy {
name = 'multi-stage';
weight = 0.7;
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
// Stage 1: Content enhancement
const enhanced = await this.enhanceContent(candidate);
// Stage 2: Structure optimization
const structured = await this.optimizeStructure(enhanced);
// Stage 3: Context adaptation
const adapted = await this.adaptToContext(structured, context);
// Stage 4: Validation and refinement
const refined = await this.refineAndValidate(adapted);
return [refined];
}
private async enhanceContent(candidate: PromptCandidate): Promise<PromptCandidate> {
// Enhance content quality and clarity
return candidate; // Implementation details
}
private async optimizeStructure(candidate: PromptCandidate): Promise<PromptCandidate> {
// Optimize prompt structure and flow
return candidate; // Implementation details
}
}
```
### Collaborative Strategies
Strategies that work together:
```typescript
class CollaborativeStrategyOrchestrator {
async orchestrateStrategies(
candidate: PromptCandidate,
context: TaskContext,
strategies: MutationStrategy[]
): Promise<PromptCandidate[]> {
const collaborationPlan = this.planCollaboration(strategies, context);
const results: PromptCandidate[] = [];
for (const phase of collaborationPlan.phases) {
const phaseResults = await this.executePhase(candidate, phase);
results.push(...phaseResults);
// Use best result as input for next phase
if (phaseResults.length > 0) {
candidate = this.selectBestCandidate(phaseResults);
}
}
return results;
}
private planCollaboration(
strategies: MutationStrategy[],
context: TaskContext
): CollaborationPlan {
// Plan how strategies should work together
return {
phases: [
{ strategies: ['reflective'], parallel: false },
{ strategies: ['semantic', 'domain-specific'], parallel: true },
{ strategies: ['validation'], parallel: false }
]
};
}
}
```
## Performance Optimization
### Strategy Caching
```typescript
class CachedMutationStrategy implements MutationStrategy {
private cache = new Map<string, PromptCandidate[]>();
async mutate(
candidate: PromptCandidate,
context: TaskContext
): Promise<PromptCandidate[]> {
const cacheKey = this.createCacheKey(candidate, context);
if (this.cache.has(cacheKey)) {
return this.cache.get(cacheKey)!;
}
const results = await this.performMutation(candidate, context);
this.cache.set(cacheKey, results);
return results;
}
private createCacheKey(candidate: PromptCandidate, context: TaskContext): string {
return `${this.hashContent(candidate.content)}-${context.category}-${context.difficulty}`;
}
}
```
### Parallel Strategy Execution
```typescript
class ParallelStrategyExecutor {
async executeStrategiesInParallel(
candidate: PromptCandidate,
context: TaskContext,
strategies: MutationStrategy[]
): Promise<PromptCandidate[]> {
const executionPromises = strategies.map(async strategy => {
try {
return await strategy.mutate(candidate, context);
} catch (error) {
this.logStrategyError(strategy.name, error);
return [];
}
});
const results = await Promise.allSettled(executionPromises);
return results
.filter(result => result.status === 'fulfilled')
.flatMap(result => (result as PromiseFulfilledResult<PromptCandidate[]>).value);
}
}
```
## Testing Custom Strategies
### Strategy Unit Tests
```typescript
describe('CustomBusinessLogicStrategy', () => {
let strategy: CustomBusinessLogicStrategy;
let mockLLMAdapter: jest.Mocked<LLMAdapter>;
beforeEach(() => {
mockLLMAdapter = createMockLLMAdapter();
strategy = new CustomBusinessLogicStrategy(
'customer-service',
testBusinessRules,
mockLLMAdapter
);
});
it('should apply business rules correctly', async () => {
const candidate = createTestCandidate();
const context = createTestContext({ category: 'customer-service' });
const mutations = await strategy.mutate(candidate, context);
expect(mutations).toHaveLength(testBusinessRules.length);
expect(mutations[0].content).toContain('customer satisfaction');
});
it('should not apply when domain does not match', () => {
const candidate = createTestCandidate();
const context = createTestContext({ category: 'code-generation' });
expect(strategy.applicable(candidate, context)).toBe(false);
});
});
```
### Integration Tests
```typescript
describe('Strategy Integration', () => {
it('should integrate custom strategy with evolution engine', async () => {
const evolutionEngine = new EvolutionEngine({
// ... other dependencies
promptMutator: new PromptMutator({
customStrategies: [new CustomBusinessLogicStrategy(...)]
})
});
const result = await evolutionEngine.startEvolution({
taskDescription: 'Customer service optimization',
seedPrompt: 'You are a customer service agent.',
maxGenerations: 5
});
expect(result.bestPrompt.content).toContain('customer satisfaction');
});
});
```
## Best Practices
1. **Strategy Focus**: Each strategy should have a clear, specific purpose
2. **Performance Monitoring**: Track strategy effectiveness and adapt weights
3. **Validation**: Always validate mutations against constraints
4. **Error Handling**: Gracefully handle strategy failures
5. **Documentation**: Clearly document strategy behavior and applicability
6. **Testing**: Comprehensive unit and integration tests
7. **Caching**: Cache expensive operations when appropriate
8. **Parallel Execution**: Use parallel execution for independent strategies
This completes the custom mutation strategies documentation. The system provides a flexible framework for implementing domain-specific optimization techniques while maintaining performance and reliability.