Katamari MCP Server

# Katamari MCP Server - ACP-Enabled Adaptive & Growing MCP Server ## Overview An adaptive MCP (Model Context Protocol) server with ACP (Agent Control Protocol) capabilities that grows and evolves based on user needs, featuring intelligent routing via a tiny LLM model, a plugin-based architecture for capabilities, and a meta-agent layer for self-modification and agency. ## Core Architecture ### 1. Intelligent Router - **Tiny LLM Model**: Small, efficient model for routing calls to appropriate services - **Context Preservation**: Minimizes context window usage for calling LLM instances - **Dynamic Discovery**: Capabilities can be queried and discovered at runtime ### 2. Capability System - **Modular Skills**: Each capability is a separate, independently testable module - **Environment Isolation**: Each capability uses its own environment (venv for Python, etc.) - **Progressive Disclosure**: Basic capability listing, detailed queries for usage info ### 3. Heuristic-Based ACP Meta-Agent Layer - **Self-Inspection**: Analyze current capabilities and usage patterns - **Dynamic Skill Generation**: Create new capabilities based on user needs - **Heuristic Governance**: Tag-based decision making for safety and approval - **Capability Composition**: Combine existing tools into new workflows - **Self-Healing**: Detect and fix issues in existing capabilities - **Stateless Design**: ACP and mini-LLM are stateless with error history access - **Parallel Testing**: Sandbox environment for testing core modifications ### 4. Development Framework - **Hot Reload**: Ability to restart server mid-development after test completion - **Testing Infrastructure**: Built-in testing for extensions and capabilities - **Documentation Generation**: Auto-generated docs at useful density levels - **Skill Registry**: Persistent storage of capability schemas and metadata ### 5. Safety & Trust - **Package Validation**: Tooling to prevent bad packages from entering ecosystem - **Code Quality Gates**: Automated checks for code quality and security - **Trust Preservation**: Mechanisms to maintain trust in the growing ecosystem - **Heuristic Governance**: Tag-based decision making for all ACP operations - **Parallel Testing**: Safe sandbox for testing core component changes - **Git-Based Tracking**: Local git tracking of all ACP-generated changes ## Implementation Phases ### Phase 1: Core Infrastructure 1. Basic MCP server skeleton 2. Tiny LLM router integration 3. Capability discovery system 4. Environment isolation framework ### Phase 2: ACP Meta-Agent 1. ACP controller implementation 2. Self-inspection and analysis tools 3. Dynamic capability generation system 4. Skill composition and workflow creation 5. Usage pattern tracking and learning ### Phase 3: Development Tools 1. Hot reload mechanism 2. Testing framework 3. Documentation generator 4. Package validation system 5. Skill registry and persistence ### Phase 4: Ecosystem Growth 1. Initial capability set (examples) 2. Developer onboarding tools 3. Community contribution guidelines 4. Trust and reputation system 5. ACP governance and approval workflows ## Key Design Principles - **Modularity**: Everything is a capability/plugin - **Isolation**: Each capability runs in its own environment - **Discoverability**: Capabilities can be found and understood incrementally - **Agency**: System can modify, compose, and grow its own capabilities - **Safety**: Built-in protection against malicious/bad code - **Memory**: System learns from usage and remembers its evolution - **Governance**: Self-modification follows rules and approval layers - **Developer Experience**: Easy to extend, test, and reload ## Technical Stack Considerations - **Router**: Tiny LLM (Qwen2-0.5B or similar) - **Core**: Python with async support - **Package Management**: Poetry/pip with virtual environments - **Testing**: pytest with capability isolation - **Documentation**: Markdown with auto-generation ## Implementation Details (Based on User Requirements) ### LLM Router - **Model**: Qwen2-0.5B (with documentation for adaptability) - **Error Reporting**: Uses error channel to provide detailed failure information to caller ### Initial Capabilities (Non-overlapping with OpenCode) 1. **Tokenless Web Search & Scraping**: Web access without API tokens 2. **Context7 Integration**: Leverage existing Context7 environment 3. **SKILLS.md Understanding**: Parse and understand skills documentation 4. **External Skill Tracking**: Track skills outside of coding editor environment ### ACP Meta-Capabilities 1. **Skill Writer**: Generate new capability modules with heuristic tags 2. **Pattern Detector**: Analyze usage patterns and suggest optimizations 3. **Capability Composer**: Combine existing tools into new workflows 4. **Self-Healer**: Detect and repair issues in existing capabilities 5. **Heuristic Engine**: Tag-based decision making for all operations 6. **Parallel Tester**: Sandbox environment for safe capability testing 7. **Git Tracker**: Local git integration for change tracking and rollbacks ### Security & Validation - **Sandboxed Execution**: Each capability runs in isolated environment - **MCP Integration**: Add MCPs for Static and Dynamic code analysis - **Asset Origin Tracking**: Critical for tracking source of each component ### Development Workflow - **Test-First Reload**: Show test results → operator review → production reload - **User-Controlled**: Manual trigger preference with automation options ### Release Strategy - **Core Build**: Essential components only - **Full Build**: Kitchen sink approach - **Dehydrated Build**: Source rebuild capability from git repos - **Asset Provenance**: Complete origin tracking for all components ### Performance & Behavior - **Startup Time**: 45 seconds acceptable for heavy environments - **Stateless Default**: With opt-in state for specific capabilities - **Fail Fast**: Immediate failure with detailed error reporting via router ## ACP Implementation Architecture ### Heuristic-Based Meta-Agent Components ```python # ACP Controller - orchestrates self-modification with heuristics class ACPController: - inspect_capabilities() # Analyze current state - propose_capability() # Design new tools with heuristic tags - validate_capability() # Test before integration - register_capability() # Safe integration with git tracking - compose_workflow() # Combine existing tools - heal_system() # Auto-repair capabilities - apply_heuristics() # Tag-based decision making # Heuristic Engine - intelligent decision making class HeuristicEngine: - evaluate_operation() # Apply heuristics to decisions - assign_tags() # Tag operations with risk/complexity - check_permissions() # Heuristic-based approval logic - learn_from_feedback() # Update heuristics based on outcomes # Parallel Testing Environment - safe sandbox class ParallelTester: - create_sandbox() # Isolated environment for testing - test_capability() # Safe testing of new/modified capabilities - validate_core_changes() # Test core router/security changes - promote_to_production() # Move tested changes to main environment # Git Tracker - version control for ACP changes class GitTracker: - commit_acp_changes() # Commit ACP-generated changes - tag_with_metadata() # Add heuristic metadata to commits - rollback_changes() # Revert problematic changes - track_evolution() # Maintain full change history ``` ### Heuristic Tag System ```python # Operation tags for decision making HEURISTIC_TAGS = { 'risk_level': ['low', 'medium', 'high', 'critical'], 'complexity': ['simple', 'moderate', 'complex', 'architectural'], 'scope': ['capability', 'workflow', 'core', 'security'], 'persistence': ['ephemeral', 'session', 'persistent', 'permanent'], 'testing_required': ['none', 'basic', 'comprehensive', 'parallel'], 'approval_needed': ['auto', 'notify', 'confirm', 'manual'] } # Heuristic rules engine def apply_heuristics(operation): risk = assess_risk(operation) complexity = assess_complexity(operation) scope = assess_scope(operation) return { 'can_auto_approve': risk == 'low' and complexity == 'simple', 'needs_parallel_test': scope in ['core', 'security'], 'requires_git_commit': scope != 'ephemeral', 'approval_level': determine_approval(risk, complexity, scope) } ``` ## Heuristic-Based ACP Design Decisions ### 1. **Heuristic Governance System** ✅ - **Tag-based decisions**: Risk level, complexity, scope, persistence tags - **Dynamic approval**: Auto-approve low-risk/simple operations, require review for high-risk - **Context-aware**: Heuristics adapt based on system state and user feedback ### 2. **Git-Based Skill Persistence** ✅ - **Local git tracking**: All ACP-generated changes committed with metadata - **Heuristic limitations**: Tags determine persistence level and rollback options - **Version control**: Natural evolution tracking and branching for experiments ### 3. **Stateless ACP with Error History** ✅ - **Stateless design**: ACP and mini-LLM maintain no persistent state - **Error access**: Queryable error history at various detail levels - **Pattern recognition**: Stateless analysis of historical error patterns ### 4. **Capability Discovery & Composition** ✅ - **Introspection API**: LLM can query available components and capabilities - **Progressive disclosure**: Basic listing → detailed usage → composition options - **Future compositions**: Phase 2 feature for workflow shortcuts and macros ### 5. **Parallel Testing Environment** ✅ - **Sandbox testing**: Isolated environment for testing core modifications - **Safe promotion**: Tested changes promoted to production after validation - **Heuristic gates**: Core changes (router, security) always require parallel testing ## Final Heuristic Design Decisions ✅ ### 1. **Heuristic Tag System** - Detailed (8-10 tags) ``` risk: low|medium|high|critical complexity: simple|moderate|complex|architectural scope: capability|workflow|core|security persistence: ephemeral|session|persistent|permanent testing: none|basic|comprehensive|parallel approval: auto|notify|confirm|manual ``` *Review and optimize low-ROI tags later* ### 2. **Heuristic Learning Strategy** - Static → Adaptive - **Phase 1**: Static heuristics with predefined rules - **Phase 2**: Add adaptive learning for new components - **Future**: ML-enhanced heuristics based on accumulated data ### 3. **Parallel Testing Environment** - Comprehensive - Full system clone with realistic data/workloads - Multiple system states and edge cases - Isolated from production environment ### 4. **Git Strategy** - Branching Model - Feature branches for ACP experiments - Merge to main after heuristic validation - ACP metadata in commit messages for traceability ## Implementation Priority ### Phase 1: Static Heuristics Foundation 1. Implement core ACP controller with static heuristics 2. Build parallel testing environment 3. Create git branching integration 4. Develop capability introspection API ### Phase 2: Adaptive Learning Integration ✅ COMPLETED 1. ✅ Add heuristic learning mechanisms for new components 2. ✅ Implement usage pattern analysis 3. ✅ Create feedback loops for heuristic refinement 4. ✅ Develop adaptive decision making #### Phase 2 Implementation Details **Adaptive Learning Engine** (`acp/adaptive_learning.py`) - Execution feedback collection and analysis - Heuristic accuracy tracking and improvement - Dynamic adjustment recommendations with confidence scoring - Learning sessions for batch processing - Performance-based heuristic optimization **Feedback Collection System** (`acp/feedback.py`) - Multi-channel feedback (user, automatic, monitoring, tests) - Feedback validation and quality control - Asynchronous feedback processing queue - Feedback analytics and trend analysis - User satisfaction tracking (1-5 ratings + comments) **Performance Tracking** (`acp/performance_tracker.py`) - Real-time system monitoring (CPU, memory, disk, network) - Execution metrics and success rate tracking - Capability health scoring (0-100 with A-F grades) - Performance trend analysis and anomaly detection - Resource usage optimization recommendations **Data Models** (`acp/data_models.py`) - Centralized data structures for all Phase 2 components - Comprehensive validation functions - JSON serialization/deserialization helpers - Learning records and adaptation proposals - Feedback events and performance snapshots **Router Integration** (`router/intelligent_router.py`) - New MCP endpoints for feedback and performance - `acp_feedback_submit` - Submit user feedback - `acp_feedback_summary` - Get feedback analytics - `acp_performance_metrics` - Get capability performance - `acp_learning_summary` - Get adaptive learning progress **Controller Updates** (`acp/controller.py`) - Integration with adaptive learning components - Feedback collection during capability execution - Performance tracking for all ACP operations - Learning engine integration for heuristic optimization **Comprehensive Testing** (`tests/test_adaptive_learning.py`) - Unit tests for all adaptive learning components - Integration tests for feedback loops - Performance tracking validation - End-to-end adaptive learning cycle tests ### Phase 3: Advanced ACP Features 1. Capability composition and workflows 2. Advanced pattern recognition 3. Cross-component optimization 4. Community contribution tools