MCP Standards

--- name: "AgentDB Advanced Features" description: "Master advanced AgentDB features including QUIC synchronization, multi-database management, custom distance metrics, hybrid search, and distributed systems integration. Use when building distributed AI systems, multi-agent coordination, or advanced vector search applications." --- # AgentDB Advanced Features ## What This Skill Does Covers advanced AgentDB capabilities for distributed systems, multi-database coordination, custom distance metrics, hybrid search (vector + metadata), QUIC synchronization, and production deployment patterns. Enables building sophisticated AI systems with sub-millisecond cross-node communication and advanced search capabilities. **Performance**: <1ms QUIC sync, hybrid search with filters, custom distance metrics. ## Prerequisites - Node.js 18+ - AgentDB v1.0.7+ (via agentic-flow) - Understanding of distributed systems (for QUIC sync) - Vector search fundamentals --- ## QUIC Synchronization ### What is QUIC Sync? QUIC (Quick UDP Internet Connections) enables sub-millisecond latency synchronization between AgentDB instances across network boundaries with automatic retry, multiplexing, and encryption. **Benefits**: - <1ms latency between nodes - Multiplexed streams (multiple operations simultaneously) - Built-in encryption (TLS 1.3) - Automatic retry and recovery - Event-based broadcasting ### Enable QUIC Sync ```typescript import { createAgentDBAdapter } from 'agentic-flow/reasoningbank'; // Initialize with QUIC synchronization const adapter = await createAgentDBAdapter({ dbPath: '.agentdb/distributed.db', enableQUICSync: true, syncPort: 4433, syncPeers: [ '192.168.1.10:4433', '192.168.1.11:4433', '192.168.1.12:4433', ], }); // Patterns automatically sync across all peers await adapter.insertPattern({ // ... pattern data }); // Available on all peers within ~1ms ``` ### QUIC Configuration ```typescript const adapter = await createAgentDBAdapter({ enableQUICSync: true, syncPort: 4433, // QUIC server port syncPeers: ['host1:4433'], // Peer addresses syncInterval: 1000, // Sync interval (ms) syncBatchSize: 100, // Patterns per batch maxRetries: 3, // Retry failed syncs compression: true, // Enable compression }); ``` ### Multi-Node Deployment ```bash # Node 1 (192.168.1.10) AGENTDB_QUIC_SYNC=true \ AGENTDB_QUIC_PORT=4433 \ AGENTDB_QUIC_PEERS=192.168.1.11:4433,192.168.1.12:4433 \ node server.js # Node 2 (192.168.1.11) AGENTDB_QUIC_SYNC=true \ AGENTDB_QUIC_PORT=4433 \ AGENTDB_QUIC_PEERS=192.168.1.10:4433,192.168.1.12:4433 \ node server.js # Node 3 (192.168.1.12) AGENTDB_QUIC_SYNC=true \ AGENTDB_QUIC_PORT=4433 \ AGENTDB_QUIC_PEERS=192.168.1.10:4433,192.168.1.11:4433 \ node server.js ``` --- ## Distance Metrics ### Cosine Similarity (Default) Best for normalized vectors, semantic similarity: ```bash # CLI npx agentdb@latest query ./vectors.db "[0.1,0.2,...]" -m cosine # API const result = await adapter.retrieveWithReasoning(queryEmbedding, { metric: 'cosine', k: 10, }); ``` **Use Cases**: - Text embeddings (BERT, GPT, etc.) - Semantic search - Document similarity - Most general-purpose applications **Formula**: `cos(θ) = (A · B) / (||A|| × ||B||)` **Range**: [-1, 1] (1 = identical, -1 = opposite) ### Euclidean Distance (L2) Best for spatial data, geometric similarity: ```bash # CLI npx agentdb@latest query ./vectors.db "[0.1,0.2,...]" -m euclidean # API const result = await adapter.retrieveWithReasoning(queryEmbedding, { metric: 'euclidean', k: 10, }); ``` **Use Cases**: - Image embeddings - Spatial data - Computer vision - When vector magnitude matters **Formula**: `d = √(Σ(ai - bi)²)` **Range**: [0, ∞] (0 = identical, ∞ = very different) ### Dot Product Best for pre-normalized vectors, fast computation: ```bash # CLI npx agentdb@latest query ./vectors.db "[0.1,0.2,...]" -m dot # API const result = await adapter.retrieveWithReasoning(queryEmbedding, { metric: 'dot', k: 10, }); ``` **Use Cases**: - Pre-normalized embeddings - Fast similarity computation - When vectors are already unit-length **Formula**: `dot = Σ(ai × bi)` **Range**: [-∞, ∞] (higher = more similar) ### Custom Distance Metrics ```typescript // Implement custom distance function function customDistance(vec1: number[], vec2: number[]): number { // Weighted Euclidean distance const weights = [1.0, 2.0, 1.5, ...]; let sum = 0; for (let i = 0; i < vec1.length; i++) { sum += weights[i] * Math.pow(vec1[i] - vec2[i], 2); } return Math.sqrt(sum); } // Use in search (requires custom implementation) ``` --- ## Hybrid Search (Vector + Metadata) ### Basic Hybrid Search Combine vector similarity with metadata filtering: ```typescript // Store documents with metadata await adapter.insertPattern({ id: '', type: 'document', domain: 'research-papers', pattern_data: JSON.stringify({ embedding: documentEmbedding, text: documentText, metadata: { author: 'Jane Smith', year: 2025, category: 'machine-learning', citations: 150, } }), confidence: 1.0, usage_count: 0, success_count: 0, created_at: Date.now(), last_used: Date.now(), }); // Hybrid search: vector similarity + metadata filters const result = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'research-papers', k: 20, filters: { year: { $gte: 2023 }, // Published 2023 or later category: 'machine-learning', // ML papers only citations: { $gte: 50 }, // Highly cited }, }); ``` ### Advanced Filtering ```typescript // Complex metadata queries const result = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'products', k: 50, filters: { price: { $gte: 10, $lte: 100 }, // Price range category: { $in: ['electronics', 'gadgets'] }, // Multiple categories rating: { $gte: 4.0 }, // High rated inStock: true, // Available tags: { $contains: 'wireless' }, // Has tag }, }); ``` ### Weighted Hybrid Search Combine vector and metadata scores: ```typescript const result = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'content', k: 20, hybridWeights: { vectorSimilarity: 0.7, // 70% weight on semantic similarity metadataScore: 0.3, // 30% weight on metadata match }, filters: { category: 'technology', recency: { $gte: Date.now() - 30 * 24 * 3600000 }, // Last 30 days }, }); ``` --- ## Multi-Database Management ### Multiple Databases ```typescript // Separate databases for different domains const knowledgeDB = await createAgentDBAdapter({ dbPath: '.agentdb/knowledge.db', }); const conversationDB = await createAgentDBAdapter({ dbPath: '.agentdb/conversations.db', }); const codeDB = await createAgentDBAdapter({ dbPath: '.agentdb/code.db', }); // Use appropriate database for each task await knowledgeDB.insertPattern({ /* knowledge */ }); await conversationDB.insertPattern({ /* conversation */ }); await codeDB.insertPattern({ /* code */ }); ``` ### Database Sharding ```typescript // Shard by domain for horizontal scaling const shards = { 'domain-a': await createAgentDBAdapter({ dbPath: '.agentdb/shard-a.db' }), 'domain-b': await createAgentDBAdapter({ dbPath: '.agentdb/shard-b.db' }), 'domain-c': await createAgentDBAdapter({ dbPath: '.agentdb/shard-c.db' }), }; // Route queries to appropriate shard function getDBForDomain(domain: string) { const shardKey = domain.split('-')[0]; // Extract shard key return shards[shardKey] || shards['domain-a']; } // Insert to correct shard const db = getDBForDomain('domain-a-task'); await db.insertPattern({ /* ... */ }); ``` --- ## MMR (Maximal Marginal Relevance) Retrieve diverse results to avoid redundancy: ```typescript // Without MMR: Similar results may be redundant const standardResults = await adapter.retrieveWithReasoning(queryEmbedding, { k: 10, useMMR: false, }); // With MMR: Diverse, non-redundant results const diverseResults = await adapter.retrieveWithReasoning(queryEmbedding, { k: 10, useMMR: true, mmrLambda: 0.5, // Balance relevance (0) vs diversity (1) }); ``` **MMR Parameters**: - `mmrLambda = 0`: Maximum relevance (may be redundant) - `mmrLambda = 0.5`: Balanced (default) - `mmrLambda = 1`: Maximum diversity (may be less relevant) **Use Cases**: - Search result diversification - Recommendation systems - Avoiding echo chambers - Exploratory search --- ## Context Synthesis Generate rich context from multiple memories: ```typescript const result = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'problem-solving', k: 10, synthesizeContext: true, // Enable context synthesis }); // ContextSynthesizer creates coherent narrative console.log('Synthesized Context:', result.context); // "Based on 10 similar problem-solving attempts, the most effective // approach involves: 1) analyzing root cause, 2) brainstorming solutions, // 3) evaluating trade-offs, 4) implementing incrementally. Success rate: 85%" console.log('Patterns:', result.patterns); // Extracted common patterns across memories ``` --- ## Production Patterns ### Connection Pooling ```typescript // Singleton pattern for shared adapter class AgentDBPool { private static instance: AgentDBAdapter; static async getInstance() { if (!this.instance) { this.instance = await createAgentDBAdapter({ dbPath: '.agentdb/production.db', quantizationType: 'scalar', cacheSize: 2000, }); } return this.instance; } } // Use in application const db = await AgentDBPool.getInstance(); const results = await db.retrieveWithReasoning(queryEmbedding, { k: 10 }); ``` ### Error Handling ```typescript async function safeRetrieve(queryEmbedding: number[], options: any) { try { const result = await adapter.retrieveWithReasoning(queryEmbedding, options); return result; } catch (error) { if (error.code === 'DIMENSION_MISMATCH') { console.error('Query embedding dimension mismatch'); // Handle dimension error } else if (error.code === 'DATABASE_LOCKED') { // Retry with exponential backoff await new Promise(resolve => setTimeout(resolve, 100)); return safeRetrieve(queryEmbedding, options); } throw error; } } ``` ### Monitoring and Logging ```typescript // Performance monitoring const startTime = Date.now(); const result = await adapter.retrieveWithReasoning(queryEmbedding, { k: 10 }); const latency = Date.now() - startTime; if (latency > 100) { console.warn('Slow query detected:', latency, 'ms'); } // Log statistics const stats = await adapter.getStats(); console.log('Database Stats:', { totalPatterns: stats.totalPatterns, dbSize: stats.dbSize, cacheHitRate: stats.cacheHitRate, avgSearchLatency: stats.avgSearchLatency, }); ``` --- ## CLI Advanced Operations ### Database Import/Export ```bash # Export with compression npx agentdb@latest export ./vectors.db ./backup.json.gz --compress # Import from backup npx agentdb@latest import ./backup.json.gz --decompress # Merge databases npx agentdb@latest merge ./db1.sqlite ./db2.sqlite ./merged.sqlite ``` ### Database Optimization ```bash # Vacuum database (reclaim space) sqlite3 .agentdb/vectors.db "VACUUM;" # Analyze for query optimization sqlite3 .agentdb/vectors.db "ANALYZE;" # Rebuild indices npx agentdb@latest reindex ./vectors.db ``` --- ## Environment Variables ```bash # AgentDB configuration AGENTDB_PATH=.agentdb/reasoningbank.db AGENTDB_ENABLED=true # Performance tuning AGENTDB_QUANTIZATION=binary # binary|scalar|product|none AGENTDB_CACHE_SIZE=2000 AGENTDB_HNSW_M=16 AGENTDB_HNSW_EF=100 # Learning plugins AGENTDB_LEARNING=true # Reasoning agents AGENTDB_REASONING=true # QUIC synchronization AGENTDB_QUIC_SYNC=true AGENTDB_QUIC_PORT=4433 AGENTDB_QUIC_PEERS=host1:4433,host2:4433 ``` --- ## Troubleshooting ### Issue: QUIC sync not working ```bash # Check firewall allows UDP port 4433 sudo ufw allow 4433/udp # Verify peers are reachable ping host1 # Check QUIC logs DEBUG=agentdb:quic node server.js ``` ### Issue: Hybrid search returns no results ```typescript // Relax filters const result = await adapter.retrieveWithReasoning(queryEmbedding, { k: 100, // Increase k filters: { // Remove or relax filters }, }); ``` ### Issue: Memory consolidation too aggressive ```typescript // Disable automatic optimization const result = await adapter.retrieveWithReasoning(queryEmbedding, { optimizeMemory: false, // Disable auto-consolidation k: 10, }); ``` --- ## Learn More - **QUIC Protocol**: docs/quic-synchronization.pdf - **Hybrid Search**: docs/hybrid-search-guide.md - **GitHub**: https://github.com/ruvnet/agentic-flow/tree/main/packages/agentdb - **Website**: https://agentdb.ruv.io --- **Category**: Advanced / Distributed Systems **Difficulty**: Advanced **Estimated Time**: 45-60 minutes