INTEGRATION-GUIDE.md•25.6 kB
# MCP Self-Learning Server - System Integration Guide
This comprehensive guide covers how to integrate the MCP Self-Learning Server with various system components, particularly Claudia (Voice Assistant) and Claudio (AIMCP Orchestrator).
## 📋 Table of Contents
- [Quick Start](#quick-start)
- [Global Installation](#global-installation)
- [System Architecture](#system-architecture)
- [Claudio Integration](#claudio-integration)
- [Claudia Integration](#claudia-integration)
- [REST API Usage](#rest-api-usage)
- [Client Libraries](#client-libraries)
- [Configuration](#configuration)
- [Deployment](#deployment)
- [Monitoring & Troubleshooting](#monitoring--troubleshooting)
## 🚀 Quick Start
### 1. Install the Package Globally
```bash
cd /home/ben/saralegui-solutions-llc/shared/MCPSelfLearningServer
npm install
npm run install-global # Creates global 'mcp-learn' command
```
### 2. Start the Services
```bash
# Start the MCP server
mcp-learn start
# In another terminal, start the REST API server
mcp-learn api
```
### 3. Verify Installation
```bash
# Run health check
mcp-learn health
# Check status
mcp-learn status
# Monitor in real-time
mcp-learn monitor
```
### 4. Test Basic Functionality
```bash
# Analyze a pattern
mcp-learn analyze --type "test_interaction" --input "hello" --output "world" --success
# Get insights
mcp-learn insights
# Export knowledge
mcp-learn export --format json
```
## 📦 Global Installation
The MCP Self-Learning Server can be installed globally for system-wide access:
### NPM Package Installation
```bash
# Install dependencies
npm install
# Link globally
npm run install-global
# Verify global installation
which mcp-learn
mcp-learn --help
```
### Systemd Service Setup
```bash
# Copy service files
sudo cp systemd/mcp-self-learning.service /etc/systemd/system/
sudo cp systemd/mcp-learning-api.service /etc/systemd/system/
# Enable services
sudo systemctl enable mcp-self-learning.service
sudo systemctl enable mcp-learning-api.service
# Start services
sudo systemctl start mcp-self-learning.service
sudo systemctl start mcp-learning-api.service
# Check status
sudo systemctl status mcp-self-learning.service
```
### Environment Configuration
Create `/etc/mcp-learning/config.json`:
```json
{
"server": {
"port": 8765,
"host": "localhost",
"autoStart": true
},
"learning": {
"maxMemorySize": 1000,
"autoSaveInterval": 300000,
"persistenceEnabled": true
},
"integrations": {
"claudio": {
"enabled": true,
"path": "/home/ben/saralegui-solutions-llc/claude-assistant"
},
"claudia": {
"enabled": true,
"path": "/home/ben/claudia"
}
}
}
```
## 🏗️ System Architecture
### Component Overview
```
┌─────────────────────────────────────────────────────────────┐
│ MCP Self-Learning Server │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │
│ │ Learning Engine │ │ Knowledge Sync │ │ API Server │ │
│ │ - Pattern Rec. │ │ - Cross-Service│ │ - REST API │ │
│ │ - Confidence │ │ - File Storage │ │ - WebSocket │ │
│ │ - Memory Mgmt │ │ - Auto-sync │ │ - Auth │ │
│ └─────────────────┘ └─────────────────┘ └──────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ Integration Layer │
├─────────────────────┬───────────────────────────────────────┤
│ Claudio MCP │ Claudia Voice Assistant │
│ - Agent Registry │ - Voice Learning Plugin │
│ - Workflow Opt. │ - Intent Recognition │
│ - Performance │ - Response Optimization │
│ - Predictions │ - Conversation Suggestions │
└─────────────────────┴───────────────────────────────────────┘
```
### Data Flow
1. **Learning Input**: Interactions from Claudio/Claudia → Learning Engine
2. **Pattern Analysis**: Learning Engine → Feature Extraction → Confidence Scoring
3. **Knowledge Storage**: Patterns → Local Storage + Shared Knowledge Directory
4. **Cross-Service Sync**: Shared Knowledge → Other Services
5. **Optimization**: Insights → Recommendations → Service Improvements
## 🤖 Claudio Integration
### Overview
The Claudio integration provides learning capabilities to the multi-agent orchestration system, enabling:
- **Agent Performance Learning**: Track and optimize agent interactions
- **Workflow Optimization**: Learn from workflow outcomes
- **Predictive Suggestions**: Predict next best actions
- **Cross-Agent Knowledge**: Share learnings between agents
### Setup
1. **Install the Learning Agent**:
```javascript
// In Claudio's agent registry
import ClaudioLearningAgent from '/home/ben/saralegui-solutions-llc/shared/MCPSelfLearningServer/integrations/claudio-integration.js';
const learningAgent = new ClaudioLearningAgent({
mcpServerUrl: 'http://localhost:8765'
});
await learningAgent.initialize();
// Register with agent registry
agentRegistry.registerAgent('learning-agent', learningAgent);
```
2. **Add MCP Tools to Claudio's Server**:
```javascript
// In mcp-claudio-server.js
import ClaudioMCPLearningTools from '/home/ben/saralegui-solutions-llc/shared/MCPSelfLearningServer/integrations/claudio-mcp-tools.js';
const learningTools = new ClaudioMCPLearningTools({
mcpServerUrl: 'http://localhost:8765'
});
// Add tools to MCP server
const toolDefinitions = learningTools.getToolDefinitions();
// Register tools...
```
### Available MCP Tools
#### `learn_from_interaction`
Learn from agent interactions:
```json
{
"agent_id": "analysis-agent",
"workflow_id": "data-processing-workflow",
"input": "Process customer data",
"output": "Processed 1000 records successfully",
"success": true,
"duration": 2500,
"metadata": {
"records_processed": 1000,
"memory_used": "50MB"
}
}
```
#### `get_learning_insights`
Get learning analytics:
```json
{
"workflow_id": "optional-workflow-filter",
"agent_id": "optional-agent-filter"
}
```
#### `optimize_workflow`
Get workflow optimization suggestions:
```json
{
"workflow_id": "data-processing-workflow",
"involved_agents": ["data-agent", "analysis-agent"],
"current_step": "validation",
"performance": {
"average_duration": 5000,
"success_rate": 0.95
}
}
```
#### `predict_next_action`
Predict next best action:
```json
{
"current_agent": "data-agent",
"workflow_state": {
"completed_steps": ["ingest", "validate"],
"current_data": {"records": 1000}
},
"user_intent": "process_customer_data",
"available_agents": [
{"id": "analysis-agent", "capabilities": ["analyze", "summarize"]},
{"id": "export-agent", "capabilities": ["export", "format"]}
]
}
```
### Integration in Claudio Workflows
```javascript
// Example workflow with learning integration
class DataProcessingWorkflow {
async execute(request) {
const workflowId = `workflow-${Date.now()}`;
try {
// Start workflow
const startTime = Date.now();
// Execute workflow steps...
const result = await this.processData(request);
// Learn from the outcome
await this.learningAgent.handleRequest({
action: 'learn-from-outcome',
data: {
workflowId,
initialRequest: request,
finalResult: result,
success: true,
totalDuration: Date.now() - startTime,
involvedAgents: ['data-agent', 'analysis-agent'],
userSatisfaction: 0.9
}
});
return result;
} catch (error) {
// Learn from failures too
await this.learningAgent.handleRequest({
action: 'learn-from-outcome',
data: {
workflowId,
initialRequest: request,
finalResult: { error: error.message },
success: false,
totalDuration: Date.now() - startTime
}
});
throw error;
}
}
}
```
## 🎤 Claudia Integration
### Overview
The Claudia integration enhances the voice assistant with learning capabilities:
- **Voice Interaction Learning**: Learn from user conversations
- **Intent Recognition Improvement**: Continuously improve intent detection
- **Response Optimization**: Learn which responses work best
- **Predictive Assistance**: Predict user needs and intents
### Setup
1. **Install the Python Client Library**:
```bash
# Ensure the client library is accessible
export PYTHONPATH="${PYTHONPATH}:/home/ben/saralegui-solutions-llc/shared/MCPSelfLearningServer/lib"
```
2. **Initialize the Learning Plugin**:
```python
# In Claudia's main initialization
import sys
sys.path.append('/home/ben/saralegui-solutions-llc/shared/MCPSelfLearningServer/integrations')
from claudia_learning_plugin import setup_claudia_learning, create_learning_config
class ClaudiaAssistant:
async def __init__(self):
# ... existing initialization ...
# Set up learning integration
learning_config = create_learning_config(
mcp_server_url='http://localhost:8765',
learning_enabled=True,
auto_optimize=True,
prediction_enabled=True
)
self.learning_plugin = await setup_claudia_learning(self, learning_config)
async def shutdown(self):
# ... existing shutdown ...
# Shutdown learning plugin
if self.learning_plugin:
await self.learning_plugin.shutdown()
```
### Integration in Voice Interactions
```python
async def handle_voice_interaction(self, audio_input):
# Start learning session
session_id = self.learning_plugin.start_session()
try:
# Process speech to text
user_input = await self.speech_to_text(audio_input)
# Detect intent
intent_result = await self.detect_intent(user_input)
intent = intent_result['intent']
confidence = intent_result['confidence']
# Get predictions if confidence is low
if confidence < 0.7:
predictions = await self.learning_plugin.predict_user_intent(user_input)
if predictions['predictions'] and predictions['confidence'] > confidence:
# Use predicted intent
better_prediction = predictions['predictions'][0]
intent = better_prediction['intent']
confidence = better_prediction['confidence']
self.logger.info(f"Used predicted intent: {intent} (confidence: {confidence})")
# Generate response
response = await self.generate_response(user_input, intent)
# Learn from the interaction
learning_result = await self.learning_plugin.learn_from_interaction(
user_input=user_input,
assistant_response=response,
intent=intent,
confidence=confidence,
success=True, # Determine based on user satisfaction
metadata={
'audio_quality': audio_input.get('quality', 'unknown'),
'processing_time': processing_time,
'response_method': 'voice'
}
)
# Apply real-time optimizations
if learning_result.get('insights', {}).get('recommendations'):
await self._apply_recommendations(learning_result['insights']['recommendations'])
# Convert response to speech
audio_response = await self.text_to_speech(response)
return audio_response
except Exception as e:
# Learn from failures
await self.learning_plugin.learn_from_interaction(
user_input=user_input or "unknown",
assistant_response=f"Error: {str(e)}",
intent=intent or "error",
confidence=0.0,
success=False
)
raise
finally:
# End learning session
await self.learning_plugin.end_session(session_success=True)
```
### Advanced Features
#### Intent Recognition Training
```python
# Improve intent recognition with training data
training_examples = [
{'input': 'turn on the lights', 'intent': 'smart_home_control', 'confidence': 1.0},
{'input': 'what\'s the weather like', 'intent': 'weather_query', 'confidence': 1.0},
{'input': 'play some music', 'intent': 'media_control', 'confidence': 1.0}
]
result = await self.learning_plugin.improve_intent_recognition(training_examples)
print(f"Training completed: {result['examples_processed']} examples processed")
```
#### Conversation Suggestions
```python
# Get suggestions for conversation continuation
conversation_history = [
{'role': 'user', 'content': 'Hello Claudia'},
{'role': 'assistant', 'content': 'Hello! How can I help you today?'},
{'role': 'user', 'content': 'I need help with my schedule'}
]
suggestions = await self.learning_plugin.get_conversation_suggestions(
conversation_history=conversation_history,
user_profile={'preferences': ['concise_responses', 'calendar_management']}
)
for suggestion in suggestions['suggestions']:
print(f"Suggested response: {suggestion['text']} (confidence: {suggestion['confidence']})")
```
## 🌐 REST API Usage
### API Endpoints
The REST API server provides HTTP access to all learning tools:
- **Base URL**: `http://localhost:8765`
- **API Documentation**: `GET /api`
- **Health Check**: `GET /health`
- **WebSocket**: `ws://localhost:8765/ws`
### Example API Usage
#### Analyze Pattern
```bash
curl -X POST http://localhost:8765/analyze \
-H "Content-Type: application/json" \
-d '{
"interaction": {
"type": "api_test",
"input": "test input",
"output": "test output",
"success": true,
"context": {"source": "api_test"},
"performance": {"duration": 150}
}
}'
```
#### Get Insights
```bash
curl -X GET http://localhost:8765/insights
```
#### Export Knowledge
```bash
curl -X GET "http://localhost:8765/export?format=json" > knowledge_export.json
```
### WebSocket Real-time Updates
```javascript
const ws = new WebSocket('ws://localhost:8765/ws');
ws.on('message', (data) => {
const message = JSON.parse(data);
switch (message.type) {
case 'pattern_analyzed':
console.log('New pattern learned:', message.data);
break;
case 'learning_cycle_completed':
console.log('Learning cycle completed:', message.data.stats);
break;
}
});
```
## 📚 Client Libraries
### Node.js Client Library
```javascript
import SelfLearningClient from './lib/self-learning-client.js';
const client = new SelfLearningClient({
baseUrl: 'http://localhost:8765'
});
await client.connect();
// Analyze pattern
const result = await client.analyzePattern({
type: 'user_interaction',
input: 'Help me with my project',
output: 'Here are some suggestions...',
success: true,
context: { userId: '123', sessionId: 'abc' }
});
// Get insights
const insights = await client.getInsights();
console.log('Learning insights:', insights);
// Real-time monitoring
const stopMonitoring = await client.monitorLearning((error, data) => {
if (error) {
console.error('Monitoring error:', error);
} else {
console.log('Status update:', data.status);
console.log('Insights:', data.insights);
}
}, 5000); // Update every 5 seconds
// Stop monitoring after 1 minute
setTimeout(() => stopMonitoring(), 60000);
```
### Python Client Library
```python
import asyncio
from self_learning_client import SelfLearningClient
async def main():
async with SelfLearningClient('http://localhost:8765') as client:
# Analyze pattern
result = await client.analyze_pattern({
'type': 'python_interaction',
'input': 'Process data',
'output': 'Data processed successfully',
'success': True,
'context': {'language': 'python'}
})
# Get insights
insights = await client.get_insights()
print(f"Total patterns: {insights.total_patterns}")
print(f"Learning cycles: {insights.learning_cycles}")
# Wait for specific milestone
milestone_insights = await client.wait_for_learning_milestone(
lambda insights: insights.total_patterns >= 10,
timeout=60.0
)
print("Milestone reached!")
asyncio.run(main())
```
## ⚙️ Configuration
### Global Configuration
Location: `/etc/mcp-learning/config.json` or `~/.mcp-learning/config.json`
```json
{
"version": "1.0.0",
"server": {
"port": 8765,
"host": "localhost",
"autoStart": false
},
"learning": {
"maxMemorySize": 1000,
"autoSaveInterval": 300000,
"persistenceEnabled": true,
"confidenceThreshold": 0.5
},
"api": {
"authentication": false,
"corsEnabled": true,
"rateLimit": {
"enabled": false,
"max": 100,
"windowMs": 900000
}
},
"integrations": {
"claudio": {
"enabled": true,
"path": "/home/ben/saralegui-solutions-llc/claude-assistant",
"autoRegister": true
},
"claudia": {
"enabled": true,
"path": "/home/ben/claudia",
"autoLoad": true
}
},
"logging": {
"level": "info",
"console": true,
"file": true,
"maxFiles": 5,
"maxSize": "10MB"
},
"sharedKnowledge": {
"enabled": true,
"directory": "/tmp/mcp-learning-shared",
"syncInterval": 60000
}
}
```
### Environment Variables
```bash
# Server configuration
export MCP_LEARN_PORT=8765
export MCP_LEARN_HOST=localhost
# Learning configuration
export MCP_LEARN_MAX_MEMORY=1000
export MCP_LEARN_AUTO_SAVE_INTERVAL=300000
# Integration paths
export CLAUDIO_PATH=/home/ben/saralegui-solutions-llc/claude-assistant
export CLAUDIA_PATH=/home/ben/claudia
# Logging
export LOG_LEVEL=info
export LOG_CONSOLE=true
export LOG_FILE=true
# Shared knowledge
export MCP_SHARED_KNOWLEDGE_DIR=/tmp/mcp-learning-shared
```
## 🚀 Deployment
### Production Deployment
1. **Install Dependencies**:
```bash
npm install --production
```
2. **Configure Services**:
```bash
# Copy and customize service files
sudo cp systemd/*.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable mcp-self-learning.service
sudo systemctl enable mcp-learning-api.service
```
3. **Set Up Monitoring**:
```bash
# Monitor with systemd
journalctl -u mcp-self-learning.service -f
# Monitor with custom tools
mcp-learn monitor --details
```
4. **Configure Firewall** (if needed):
```bash
# Allow API port
sudo ufw allow 8765/tcp
```
### Docker Deployment
```dockerfile
# Dockerfile
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install --production
COPY . .
EXPOSE 8765
CMD ["npm", "start"]
```
```yaml
# docker-compose.yml
version: '3.8'
services:
mcp-learning:
build: .
ports:
- "8765:8765"
volumes:
- ./data:/app/data
- ./logs:/app/logs
environment:
- NODE_ENV=production
- MCP_LEARN_PORT=8765
restart: unless-stopped
```
### Kubernetes Deployment
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mcp-self-learning
spec:
replicas: 1
selector:
matchLabels:
app: mcp-self-learning
template:
metadata:
labels:
app: mcp-self-learning
spec:
containers:
- name: mcp-server
image: mcp-self-learning:latest
ports:
- containerPort: 8765
env:
- name: NODE_ENV
value: production
- name: MCP_LEARN_PORT
value: "8765"
volumeMounts:
- name: data-storage
mountPath: /app/data
- name: shared-knowledge
mountPath: /tmp/mcp-learning-shared
volumes:
- name: data-storage
persistentVolumeClaim:
claimName: mcp-data-pvc
- name: shared-knowledge
persistentVolumeClaim:
claimName: mcp-shared-pvc
---
apiVersion: v1
kind: Service
metadata:
name: mcp-learning-service
spec:
selector:
app: mcp-self-learning
ports:
- port: 8765
targetPort: 8765
type: ClusterIP
```
## 📊 Monitoring & Troubleshooting
### Health Monitoring
```bash
# Run comprehensive health check
mcp-learn health
# Check system status
mcp-learn status --verbose
# Real-time monitoring
mcp-learn monitor --details --interval 3
```
### Log Analysis
```bash
# View server logs
journalctl -u mcp-self-learning.service -f
# View API logs
journalctl -u mcp-learning-api.service -f
# View application logs
tail -f logs/mcp-server.log
# Search for errors
grep -i error logs/mcp-server.log
```
### Performance Monitoring
```bash
# Monitor resource usage
htop -p $(pgrep -f mcp-self-learning)
# Check memory usage
ps aux | grep mcp-self-learning
# Network monitoring
netstat -tlnp | grep 8765
```
### Common Issues
#### 1. Server Won't Start
**Symptoms**: `mcp-learn start` fails or service won't start
**Diagnosis**:
```bash
mcp-learn health
journalctl -u mcp-self-learning.service -n 50
```
**Solutions**:
- Check Node.js version: `node --version` (requires 18+)
- Verify port availability: `netstat -tlnp | grep 8765`
- Check permissions: `ls -la mcp-self-learning-server.js`
- Review configuration: `cat ~/.mcp-learning/config.json`
#### 2. Learning Not Working
**Symptoms**: Patterns not being learned, insights empty
**Diagnosis**:
```bash
mcp-learn insights
curl -X GET http://localhost:8765/status
```
**Solutions**:
- Check data directory permissions: `ls -la data/`
- Verify learning engine initialization: `grep "Learning engine" logs/mcp-server.log`
- Test basic pattern analysis: `mcp-learn analyze --type test --input "hello" --output "world" --success`
#### 3. Integration Issues
**Symptoms**: Claudio/Claudia integration not working
**Diagnosis**:
- Check integration paths in configuration
- Verify client library imports
- Test API connectivity: `curl http://localhost:8765/health`
**Solutions**:
- Update configuration paths
- Install missing dependencies
- Check network connectivity between services
#### 4. Performance Issues
**Symptoms**: Slow response times, high memory usage
**Diagnosis**:
```bash
mcp-learn status --verbose
curl -X GET http://localhost:8765/metrics
```
**Solutions**:
- Adjust `maxMemorySize` in configuration
- Implement pattern pruning: Learning engine automatically prunes old patterns
- Scale horizontally if needed
### Debugging
#### Enable Debug Mode
```bash
# Start in debug mode
NODE_ENV=development DEBUG=1 mcp-learn start
# Or set log level
export LOG_LEVEL=debug
mcp-learn start
```
#### API Debugging
```bash
# Test API endpoints
curl -v http://localhost:8765/api
curl -v http://localhost:8765/health
curl -v -X POST http://localhost:8765/analyze \
-H "Content-Type: application/json" \
-d '{"interaction": {"type": "test", "input": "debug", "output": "test", "success": true}}'
```
#### WebSocket Debugging
```javascript
// Test WebSocket connection
const WebSocket = require('ws');
const ws = new WebSocket('ws://localhost:8765/ws');
ws.on('open', () => console.log('Connected'));
ws.on('message', (data) => console.log('Message:', JSON.parse(data)));
ws.on('error', (error) => console.error('WebSocket error:', error));
```
### Support and Maintenance
#### Regular Maintenance Tasks
1. **Weekly**:
- Review learning insights: `mcp-learn insights`
- Check log file sizes: `du -h logs/`
- Export knowledge backup: `mcp-learn export --format json`
2. **Monthly**:
- Update dependencies: `npm audit && npm update`
- Review and rotate logs: `logrotate` or manual cleanup
- Performance analysis: `mcp-learn status --verbose`
3. **Quarterly**:
- Full health check: `mcp-learn health`
- Configuration review
- Integration testing
#### Backup and Recovery
```bash
# Backup learning data
cp -r data/ backups/data-$(date +%Y%m%d)/
cp -r logs/ backups/logs-$(date +%Y%m%d)/
# Export knowledge
mcp-learn export --output backups/knowledge-$(date +%Y%m%d).json
# Backup shared knowledge
cp -r /tmp/mcp-learning-shared/ backups/shared-$(date +%Y%m%d)/
# Restore from backup
cp -r backups/data-20250829/ data/
mcp-learn start
```
## 📞 Getting Help
- **Documentation**: This guide and `README.md`
- **Health Check**: `mcp-learn health`
- **Status Check**: `mcp-learn status --verbose`
- **Real-time Monitoring**: `mcp-learn monitor --details`
- **API Documentation**: `http://localhost:8765/api`
---
**Built with ❤️ for seamless system integration and autonomous learning**