DevOps AI Toolkit

# PRD-44: Semantic Resource Matching via Vector Database **Status**: Draft **Created**: 2025-08-05 **GitHub Issue**: [#44](https://github.com/vfarcic/dot-ai/issues/44) **Dependencies**: [PRD #43 - Dependency-Aware Resource Discovery](./43-dependency-aware-resource-discovery.md) **Related PRDs**: Enhances foundation built by PRD #43 ## Executive Summary While PRD #43 provides structured resource discovery and dependency awareness, users still miss optimal high-level abstractions because the AI cannot semantically match generic resource names to specific intents. This PRD implements Vector DB-based semantic matching to ensure users get the simplest possible solutions for their needs. ## Problem Statement ### Remaining Gaps After PRD #43 1. **Generic Resource Blindness**: AI doesn't recognize that `SQL (devopstoolkit.live)` handles "PostgreSQL" intent 2. **No Semantic Understanding**: Can't match user concepts to abstract resource capabilities 3. **Missing Solution Patterns**: No learning from successful multi-resource deployments 4. **Intent Complexity Mismatch**: Users get complex solutions when simple ones exist ### Real-World Impact - User requests "PostgreSQL database" → AI sees structured Azure resources but misses generic SQL composite - `sqls.devopstoolkit.live` provides one-resource solution but gets buried in complex alternatives - No learning from organizational deployment patterns and user preferences - Suboptimal user experience despite having better tools available ## Success Criteria ### Primary Goals - **Semantic Intent Matching**: "PostgreSQL" intent finds `sqls.devopstoolkit.live` as top candidate - **Abstraction Prioritization**: Composite resources ranked above complex multi-resource solutions - **Pattern Learning**: System learns from successful deployments and user choices - **Context Awareness**: Multi-cloud resources understood for their capabilities ### Success Metrics - Generic composite resources (SQL, App) appear in relevant recommendations - User satisfaction with solution simplicity increases - Reduced average deployment complexity for standard use cases - Improved semantic match accuracy for domain-specific resources ## Solution Architecture ### Vector Database Enhancement Layer Builds on PRD #43's structured discovery to add semantic intelligence: ```typescript // PRD #43 provides structured solutions const structuredSolutions = await dependencyAwareDiscovery(intent); // This PRD adds semantic ranking const semanticRanking = await vectorDB.rankBySemantic(intent, structuredSolutions); const finalRanking = combineStructuralAndSemantic(structuredSolutions, semanticRanking); ``` ### Semantic Knowledge Sources #### 1. Resource Capability Mapping Store enhanced metadata about resource capabilities: ```json { "resource": "SQL (devopstoolkit.live/v1beta1)", "capabilities": ["postgresql", "mysql", "database", "multi-cloud"], "providers": ["azure", "gcp", "aws"], "abstractions": ["high-availability", "persistent-storage", "backup"], "complexity": "low" } ``` #### 2. Intent Pattern Learning Learn from successful user choices and deployment patterns: ```json { "intent_pattern": "postgresql + azure + high-availability", "successful_solutions": [ { "solution": "SQL (devopstoolkit.live)", "user_satisfaction": 0.95, "deployment_success": 0.98 }, { "solution": "ResourceGroup + Server + FirewallRule", "user_satisfaction": 0.75, "deployment_success": 0.90 } ] } ``` #### 3. Cross-Domain Knowledge Store domain-specific relationships and requirements: ```json { "domain": "azure-databases", "requirements": { "foundation": ["ResourceGroup"], "security": ["FirewallRule"], "alternatives": { "simple": ["SQL (devopstoolkit.live)"], "detailed": ["Server + Database + User"] } } } ``` ## Technical Implementation ### Vector Database Schema #### Resource Embedding Store - **Resource vectors**: Semantic embeddings of resource capabilities - **Intent vectors**: User intent embeddings for similarity matching - **Solution patterns**: Multi-resource solution embeddings #### Knowledge Enhancement Pipeline ```typescript // 1. Extract semantic features from structured solutions (PRD #43) const solutions = await structuredDiscovery.getSolutions(intent); // 2. Generate embeddings for semantic matching const embeddings = await generateEmbeddings(solutions, intent); // 3. Semantic similarity scoring const semanticScores = await vectorDB.similarity(intentEmbedding, resourceEmbeddings); // 4. Combined ranking (structural + semantic) const finalRanking = combineScores(structuralScores, semanticScores); ``` ### Integration with PRD #43 #### Enhanced Solution Ranking Combines structural completeness (PRD #43) with semantic relevance: ```typescript const finalScore = (structuralScore * 0.6) + (semanticScore * 0.4); ``` #### Fallback Strategy - Primary: Semantic-enhanced recommendations - Fallback: PRD #43 structured solutions (if Vector DB unavailable) - Ultimate fallback: Current system (if both enhancements fail) ## Implementation Milestones ### Milestone 1: Resource Capability Extraction and Storage - [ ] Implement resource metadata enhancement from CRD schemas - [ ] Build Vector DB schema for resource capabilities - [ ] Create semantic embedding generation for resources - **Success Criteria**: sqls.devopstoolkit.live has rich capability metadata stored ### Milestone 2: Intent-to-Resource Semantic Matching - [ ] Implement user intent embedding generation - [ ] Build semantic similarity matching between intents and resources - [ ] Create semantic scoring integration with PRD #43's structural scoring - **Success Criteria**: "PostgreSQL" intent semantically matches SQL composite resource ### Milestone 3: Solution Pattern Learning System - [ ] Implement user choice tracking and feedback collection - [ ] Build pattern learning from successful deployments - [ ] Create adaptive ranking based on learned patterns - **Success Criteria**: System learns user preferences and improves recommendations over time ### Milestone 4: Cross-Domain Knowledge Integration - [ ] Implement domain-specific knowledge storage (Azure, GCP, AWS patterns) - [ ] Build multi-concept intent understanding (cloud + database + HA) - [ ] Create context-aware resource capability matching - **Success Criteria**: System understands that generic resources can fulfill cloud-specific intents ### Milestone 5: Production Integration and Optimization - [ ] Integrate with existing MCP recommendation pipeline - [ ] Implement performance optimization and caching - [ ] Create monitoring and feedback loop for continuous improvement - **Success Criteria**: Feature deployed with measurable improvement in recommendation quality ## Risk Assessment ### Technical Risks - **Vector DB Dependency**: Additional infrastructure complexity and potential failure points - **Embedding Quality**: Semantic embeddings may not capture resource relationships accurately - **Performance Impact**: Additional Vector DB queries may slow recommendations ### Mitigation Strategies - **Graceful Degradation**: Fall back to PRD #43 solutions if Vector DB unavailable - **Incremental Rollout**: A/B testing to validate embedding quality - **Caching Strategy**: Cache frequent intent-resource matches for performance ## Dependencies and Assumptions ### Technical Dependencies - **PRD #43 Completion**: Requires structured solution discovery as foundation - Vector DB infrastructure (Qdrant) availability - Embedding generation capability (Claude/OpenAI) ### Assumptions - Resource capability metadata can be accurately extracted/curated - User feedback collection is feasible for pattern learning - Semantic embeddings effectively capture intent-resource relationships ## Related Work ### Builds Upon - **PRD #43**: Provides structured solutions for semantic enhancement - Existing Vector DB pattern storage infrastructure - Current Claude AI integration for embeddings ### Future Enhancements - Machine learning-based user preference modeling - Automated resource capability discovery from documentation - Community-driven pattern sharing and validation ## Appendix ### Example: Enhanced PostgreSQL Discovery Flow **After PRD #43 (Structural):** ``` Intent: "PostgreSQL on Azure" → Structured solutions: 1. [ResourceGroup, Server, FirewallRule] (score: 85) 2. [StatefulSet, ConfigMap, Secret] (score: 60) → Missing: sqls.devopstoolkit.live (generic name doesn't match) ``` **After PRD #44 (Semantic Enhancement):** ``` Intent: "PostgreSQL on Azure" → Semantic matching: sqls.devopstoolkit.live matches "postgresql" + "azure" → Enhanced ranking: 1. SQL (devopstoolkit.live) (combined score: 95) 2. [ResourceGroup, Server, FirewallRule] (combined score: 85) 3. [StatefulSet, ConfigMap, Secret] (combined score: 60) ``` ### Vector DB Integration Points - **Resource Enhancement**: Add semantic metadata to structured solutions from PRD #43 - **Intent Analysis**: Understand multi-concept user intents (cloud + database + requirements) - **Pattern Learning**: Store successful solution patterns for future recommendations - **Feedback Loop**: Learn from user choices to improve semantic matching accuracy