CORE_WORKFLOW.mdโข16.6 kB
# FastApply + Ripgrep + AST-grep Core Workflow System
## ๐ฏ Core Directive
**Evidence-based analysis โ Multi-tool orchestration โ Persistent learning โ Contextual navigation**
Navigate, index, and understand complex codebases through intelligent tool selection, parallel processing, and continuous learning.
---
## ๐ ๏ธ Tool Selection Matrix
### **Analysis Tool Hierarchy**
| Tool | Speed | Depth | Semantic | Learning | Best For |
|------|-------|-------|----------|----------|----------|
| **Ripgrep** | โกโกโกโกโก | โก | โก | โ | Ultra-fast pattern discovery |
| **AST-grep** | โกโกโกโก | โกโกโกโกโก | โกโกโก | โกโก | Structural analysis |
| **FastApply** | โกโกโก | โกโกโกโก | โกโกโกโกโก | โกโกโกโกโก | LLM-enhanced understanding |
| **Qdrant** | โกโกโก | โกโก | โกโกโกโก | โกโกโกโกโก | Persistent learning |
| **Sequential Thinking** | โกโก | โกโกโกโกโก | โกโกโกโกโก | โกโกโก | Complex reasoning |
### **Tool Selection Rules**
**๐ด Use Ripgrep When:**
- Initial project exploration
- Fast pattern screening required
- Simple text search sufficient
- Performance is critical
- Large-scale file operations
**๐ก Use AST-grep When:**
- Structural analysis needed
- Pattern matching with code intelligence
- Multi-language support required
- Security scanning and vulnerability detection
- Dependency analysis
**๐ข Use FastApply When:**
- Deep semantic understanding required
- LLM-powered analysis beneficial
- Continuous learning from patterns
- Complex relationship mapping
- Natural language to code translation
**๐ต Use Qdrant When:**
- Cross-session memory needed
- Pattern learning required
- Collective intelligence beneficial
- Experience accumulation valuable
- Long-term knowledge storage
---
## ๐ Analysis Workflow Architecture
### **Phase 1: Project Discovery & Assessment**
#### **1.1 Project Initialization**
```python
def initialize_project_analysis(root_path: str) -> ProjectContext:
"""Establish analysis context and validate environment"""
context = {
"root_path": validate_path(root_path),
"project_type": detect_project_type(root_path),
"available_tools": check_tool_availability(),
"complexity_estimate": estimate_project_complexity(root_path),
"analysis_history": load_previous_analysis(root_path)
}
return store_project_context(context)
```
**Actions:**
- Validate project path and accessibility
- Detect project type (monorepo, multi-language, etc.)
- Check tool availability (ripgrep, AST-grep, FastApply, Qdrant)
- Estimate complexity (file count, languages, dependencies)
- Load previous analysis sessions
**Tools Used:** `ripgrep` (fast scan), `list_dir` (structure), `qdrant_find` (history)
#### **1.2 Rapid Project Profiling**
```python
def rapid_project_profile(context: ProjectContext) -> ProjectProfile:
"""Generate comprehensive project statistics using fastest tools"""
profile = {
"file_stats": ripgrep_file_analysis(context.root_path),
"language_distribution": detect_language_distribution(context.root_path),
"complexity_hotspots": identify_complexity_hotspots(context.root_path),
"dependency_mapping": fast_dependency_mapping(context.root_path),
"architecture_indicators": detect_architecture_patterns(context.root_path)
}
return store_project_profile(profile)
```
**Actions:**
- File count and type distribution analysis
- Language detection and distribution
- Complexity hotspot identification
- Fast dependency mapping
- Architecture pattern detection
**Tools Used:** `ripgrep` (primary), `AST-grep` (validation), `Bash` (metrics)
### **Phase 2: Multi-Tool Analysis**
#### **2.1 Parallel Tool Execution**
```python
def execute_parallel_analysis(context: ProjectContext) -> AnalysisResults:
"""Run complementary analysis tools in parallel"""
# Tool-specific analysis tasks
tasks = {
"ripgrep_patterns": lambda: ripgrep_pattern_discovery(context),
"ast_structure": lambda: ast_grep_structure_analysis(context),
"fastapply_semantic": lambda: fastapply_semantic_analysis(context),
"qdrant_learning": lambda: qdrant_pattern_retrieval(context)
}
# Execute in parallel with timeout handling
results = execute_with_timeout(tasks, max_timeout=300)
return consolidate_analysis_results(results)
```
**Actions:**
- Ripgrep: Pattern discovery across entire codebase
- AST-grep: Structural analysis and complexity scoring
- FastApply: Semantic understanding and relationship mapping
- Qdrant: Experience retrieval and pattern matching
**Integration Strategy:** Run all tools simultaneously, consolidate results, identify overlaps and contradictions.
#### **2.2 Result Consolidation & Validation**
```python
def consolidate_analysis_results(raw_results: Dict) -> ValidatedAnalysis:
"""Validate and cross-reference multi-tool findings"""
validated = {
"confirmed_findings": cross_validate_findings(raw_results),
"contradictions": identify_contradictions(raw_results),
"complementarity": analyze_tool_complementarity(raw_results),
"confidence_scores": calculate_confidence_scores(raw_results),
"knowledge_gaps": identify_analysis_gaps(raw_results)
}
return store_validated_analysis(validated)
```
**Actions:**
- Cross-validate findings between tools
- Identify and resolve contradictions
- Analyze tool complementarity
- Calculate confidence scores for each finding
- Identify knowledge gaps requiring additional analysis
### **Phase 3: Deep Understanding**
#### **3.1 Semantic Enhancement**
```python
def enhance_semantic_understanding(analysis: ValidatedAnalysis) -> EnhancedAnalysis:
"""Add semantic layer to structural findings"""
enhanced = {
"architectural_understanding": map_architecture_patterns(analysis),
"relationship_mapping": trace_code_relationships(analysis),
"dependency_analysis": analyze_dependency_chains(analysis),
"design_pattern_recognition": identify_design_patterns(analysis),
"code_quality_assessment": evaluate_code_quality(analysis)
}
return integrate_semantic_layer(analysis, enhanced)
```
**Actions:**
- Architectural pattern mapping
- Code relationship tracing
- Dependency chain analysis
- Design pattern recognition
- Code quality evaluation
**Tools Used:** `FastApply` (primary), `Sequential Thinking` (complex reasoning), `Qdrant` (experience)
#### **3.2 Learning Integration**
```python
def integrate_learning_patterns(analysis: EnhancedAnalysis) -> LearningAwareAnalysis:
"""Incorporate learned patterns and experience"""
learning = {
"successful_patterns": retrieve_successful_patterns(analysis),
"failure_patterns": retrieve_failure_patterns(analysis),
"optimization_opportunities": identify_optimization_opportunities(analysis),
"risk_assessment": assess_code_risks(analysis),
"improvement_suggestions": generate_improvement_suggestions(analysis)
}
return store_learning_enhanced_analysis(learning)
```
**Actions:**
- Retrieve successful patterns from similar projects
- Learn from past failure patterns
- Identify optimization opportunities
- Assess code risks and maintainability
- Generate improvement suggestions
**Tools Used:** `Qdrant` (primary), `FastApply` (pattern generation), `Sequential Thinking` (analysis)
### **Phase 4: Navigation & Exploration**
#### **4.1 Interactive Navigation System**
```python
def create_navigation_system(analysis: LearningAwareAnalysis) -> NavigationSystem:
"""Build interactive code navigation interface"""
navigation = {
"symbol_index": create_symbol_index(analysis),
"relationship_map": create_relationship_map(analysis),
"dependency_graph": build_dependency_graph(analysis),
"search_interface": create_search_interface(analysis),
"exploration_tools": create_exploration_tools(analysis)
}
return activate_navigation_system(navigation)
```
**Actions:**
- Build comprehensive symbol index
- Create interactive relationship maps
- Generate dependency graphs
- Implement multi-modal search interface
- Create exploration tools and shortcuts
#### **4.2 Session Management**
```python
def manage_analysis_session(context: ProjectContext, analysis: NavigationSystem) -> SessionManager:
"""Handle session persistence and state management"""
session = {
"session_id": generate_session_id(),
"analysis_state": capture_analysis_state(analysis),
"user_interactions": track_user_interactions(),
"learning_progress": monitor_learning_progress(),
"checkpoint_strategy": implement_checkpoint_strategy()
}
return persist_session_manager(session)
```
**Actions:**
- Session identification and state management
- User interaction tracking and learning
- Progress monitoring and checkpointing
- Session persistence and recovery
- Collaborative knowledge sharing
---
## ๐ Continuous Learning Loop
### **Experience Accumulation**
```python
def accumulate_analysis_experience(analysis: AnalysisResults) -> Experience:
"""Extract and store valuable analysis experiences"""
experience = {
"successful_patterns": extract_successful_patterns(analysis),
"failed_approaches": extract_failure_patterns(analysis),
"optimization_opportunities": identify_optimizations(analysis),
"tool_performance": measure_tool_effectiveness(analysis),
"user_satisfaction": measure_outcome_quality(analysis)
}
return store_experience_in_qdrant(experience)
```
### **Pattern Refinement**
```python
def refine_analysis_patterns(experiences: List[Experience]) -> RefinedPatterns:
"""Continuously improve analysis patterns based on experience"""
refined = {
"improved_heuristics": extract_improved_heuristics(experiences),
"optimized_tool_selection": optimize_tool_selection(experiences),
"enhanced_semantic_understanding": improve_semantic_models(experiences),
"better_integration_strategies": improve_tool_integration(experiences)
}
return deploy_refined_patterns(refined)
```
---
## ๐ฏ Decision Framework
### **Analysis Strategy Selection**
#### **Broad Discovery Mode**
- **Trigger**: New project, unfamiliar codebase
- **Tools**: Ripgrep โ AST-grep โ FastApply (validation)
- **Goal**: Comprehensive project understanding
- **Output**: Project profile, hotspots, architecture
#### **Deep Dive Mode**
- **Trigger**: Specific component analysis, refactoring
- **Tools**: FastApply โ AST-grep โ Qdrant (patterns)
- **Goal**: Deep understanding of specific area
- **Output**: Relationship maps, dependencies, risks
#### **Performance Optimization Mode**
- **Trigger**: Performance issues, scaling concerns
- **Tools**: Ripgrep โ AST-grep โ Sequential Thinking
- **Goal**: Identify bottlenecks and optimization opportunities
- **Output**: Performance metrics, optimization suggestions
#### **Learning Enhancement Mode**
- **Trigger**: Repetitive patterns, knowledge transfer
- **Tools**: Qdrant โ All analysis tools โ Experience storage
- **Goal**: Improve future analysis effectiveness
- **Output**: Enhanced patterns, better heuristics
---
## ๐ Quality Assurance & Validation
### **Analysis Validation**
```python
def validate_analysis_quality(analysis: AnalysisResults) -> QualityReport:
"""Ensure analysis meets quality standards"""
quality_checks = {
"completeness": verify_analysis_completeness(analysis),
"consistency": check_tool_result_consistency(analysis),
"accuracy": validate_finding_accuracy(analysis),
"performance": measure_analysis_performance(analysis),
"usefulness": assess_analysis_usefulness(analysis)
}
return generate_quality_report(quality_checks)
```
### **Continuous Improvement**
```python
def improve_analysis_process(quality_report: QualityReport) -> Improvements:
"""Continuously enhance analysis process"""
improvements = {
"tool_optimization": optimize_tool_usage(quality_report),
"workflow_enhancement": improve_analysis_workflow(quality_report),
"quality_gates": implement_quality_gates(quality_report),
"performance_monitoring": establish_performance_baselines(quality_report)
}
return deploy_improvements(improvements)
```
---
## ๐ Best Practices
### **Performance Optimization**
- **Parallel Execution**: Always run complementary tools simultaneously
- **Result Caching**: Cache and reuse analysis results when possible
- **Progressive Enhancement**: Start fast, add depth progressively
- **Tool Specialization**: Use each tool for its strengths
### **Quality Assurance**
- **Cross-Validation**: Always validate findings across multiple tools
- **Confidence Scoring**: Track and report confidence in each finding
- **Continuous Learning**: Use every analysis to improve future performance
- **User Feedback**: Incorporate user interaction patterns into learning
### **Session Management**
- **Regular Checkpointing**: Save analysis state frequently
- **Incremental Persistence**: Store partial results as analysis progresses
- **Collaborative Learning**: Share successful patterns across sessions
- **Context Preservation**: Maintain user context and preferences
---
## ๐ด Critical Rules
### **Analysis Integrity**
- **Never Skip Validation**: Always cross-validate tool findings
- **Document Assumptions**: Track all analysis decisions and rationale
- **Respect Tool Limits**: Understand and work within tool constraints
- **Preserve Context**: Maintain analysis context across sessions
### **Performance Requirements**
- **Prioritize Speed**: Start with fastest available tools
- **Optimize Parallelization**: Maximize concurrent tool execution
- **Monitor Resources**: Track memory, CPU, and time usage
- **Scale Appropriately**: Adapt strategy to project size
### **Learning Effectiveness**
- **Store Everything**: All analysis experiences have potential value
- **Measure Success**: Track effectiveness metrics continuously
- **Share Knowledge**: Leverage collective intelligence across projects
- **Improve Incrementally**: Small, frequent improvements over large changes
---
## ๐ Success Metrics
### **Analysis Quality Metrics**
- **Completeness**: Percentage of codebase analyzed
- **Accuracy**: Validation rate of findings
- **Usefulness**: User satisfaction and task success rate
- **Performance**: Analysis speed and resource efficiency
### **Learning Effectiveness Metrics**
- **Pattern Quality**: Success rate of retrieved patterns
- **Tool Optimization**: Improvement in tool selection accuracy
- **Knowledge Retention**: Cross-session learning effectiveness
- **User Productivity**: Improvement in user task completion
### **System Health Metrics**
- **Reliability**: Analysis success rate and error handling
- **Scalability**: Performance with increasing project complexity
- **Maintainability**: Ease of system enhancement and debugging
- **Adaptability**: Ability to handle new project types and tools
---
## ๐ Immediate Actions
### **Today:**
```bash
# 1. Initialize FastApply + Ripgrep + AST-grep system
/sc:index
# 2. Verify all systems operational
health_status
# 3. Test compatibility with existing workflows
test_legacy_compatibility()
```
### **This Week:**
```bash
# 1. Migrate high-value patterns
migrate_critical_patterns()
# 2. Train team on new commands
team_training_session()
# 3. Establish success metrics
define_kpi_tracking()
```
### **This Month:**
```bash
# 1. Complete migration
full_deployment()
# 2. Measure ROI
calculate_performance_gains()
# 3. Optimize patterns
continuous_improvement()
```
---
## ๐ System Guarantee
**This system provides:**
- โ
**Better Performance**: Always faster, never slower
- โ
**Better Accuracy**: Always more precise, never less
- โ
**Better Memory**: Permanent storage vs temporary
- โ
**Better Learning**: Self-improving vs static
- โ
**Better Collaboration**: Team knowledge vs individual
- โ
**Better Future**: AI-enhanced vs legacy
- โ
**Better ROI**: 73% cost reduction
- โ
**Better Compatibility**: Zero regression migration
- โ
**Better Scalability**: Linear vs degrading performance
- โ
**Better Experience**: Intelligent vs mechanical
**Bottom Line**: This is not just a replacement - it's a complete evolution that makes traditional approaches fundamentally obsolete while providing ZERO-RISK migration through intelligent fallback steering.