NCP - Natural Context Provider

# Code-Mode Phase 5: Monitoring & Audit **Status**: ✅ Complete Comprehensive audit trail for Code-Mode security events, enabling enterprise compliance and security monitoring. ## Overview Phase 5 adds security event logging to track: - Code execution (start, success, failure, timeout) - Network access (requests, permissions, denials) - Binding usage - Security violations All events are logged to `~/.ncp/audit/` in JSONL format (JSON Lines) for easy parsing and analysis. ## Architecture ``` ┌─────────────────────────────────────────────────────────┐ │ CodeExecutor / NetworkPolicyManager / BindingsManager │ │ │ │ → Code execution starts │ │ → Network request attempted │ │ → Permission requested │ │ → Binding accessed │ └──────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ AuditLogger │ │ │ │ • Event validation │ │ • Sensitive data redaction │ │ • JSONL formatting │ │ • File rotation │ └──────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ ~/.ncp/audit/audit-YYYY-MM-DD.jsonl │ │ │ │ One JSON object per line │ │ Easy to parse, grep, analyze │ └─────────────────────────────────────────────────────────┘ ``` ## Event Types ### Code Execution Events - **`code_execution_start`**: Code execution begins - **`code_execution_success`**: Code executed successfully - **`code_execution_error`**: Code execution failed - **`code_execution_timeout`**: Execution timed out ### Network Access Events - **`network_request_allowed`**: Network request permitted - **`network_request_denied`**: Network request blocked - **`network_permission_granted`**: User granted network permission - **`network_permission_denied`**: User denied network permission - **`network_permission_revoked`**: Permission manually revoked ### Binding Events - **`binding_accessed`**: Binding method called - **`binding_created`**: New binding created ### Security Events - **`security_violation`**: Security policy violated - **`prototype_pollution_blocked`**: Prototype pollution attempt blocked - **`worker_thread_failed`**: Worker thread crashed ## Event Structure Every audit event has this structure: ```json { "timestamp": "2025-11-19T10:15:30.123Z", "type": "code_execution_success", "severity": "info", "context": { "mcpName": "code-mode", "bindingName": null, "userId": null, "sessionId": "session-1732008930123-abc123" }, "details": { "codeSnippet": "console.log('Hello');\nreturn { success: true };", "resultPreview": "{\"success\":true}", "durationMs": 45 }, "outcome": "success" } ``` ### Fields - **`timestamp`**: ISO 8601 timestamp - **`type`**: Event type (enum) - **`severity`**: `info`, `warning`, `error`, or `critical` - **`context`**: Who/what triggered the event - **`details`**: Event-specific data - **`outcome`**: `success`, `failure`, `blocked`, or `pending` ## Configuration Configure audit logging via environment variables or settings file: ### Environment Variables ```bash # Enable/disable audit logging export NCP_AUDIT_ENABLED=true # Audit log directory (default: ~/.ncp/audit/) export NCP_AUDIT_DIR=/var/log/ncp/audit # Maximum file size before rotation (MB) export NCP_AUDIT_MAX_SIZE_MB=10 # Maximum number of rotated files to keep export NCP_AUDIT_MAX_FILES=5 # Include code snippets in logs (default: true) export NCP_AUDIT_INCLUDE_CODE=true # Redact sensitive data (default: true) export NCP_AUDIT_REDACT_SENSITIVE=true ``` ### Settings File Edit `~/.ncp/settings.json`: ```json { "audit": { "enabled": true, "maxFileSizeMB": 10, "maxFiles": 5, "includeCodeSnippets": true, "redactSensitiveData": true } } ``` ## Example Events ### Code Execution Success ```json { "timestamp": "2025-11-19T10:15:30.456Z", "type": "code_execution_success", "severity": "info", "context": { "mcpName": "code-mode", "sessionId": "session-1732008930123-abc123" }, "details": { "codeSnippet": "const repos = await github.list_repos({owner: 'anthropics'});\nreturn repos.length;", "resultPreview": "42", "durationMs": 1234 }, "outcome": "success" } ``` ### Network Permission Granted ```json { "timestamp": "2025-11-19T10:16:15.789Z", "type": "network_permission_granted", "severity": "info", "context": { "mcpName": "lg-remote", "bindingName": "LG Remote", "sessionId": "session-1732008930123-abc123" }, "details": { "url": "http://192.168.1.100:3000/status", "permanent": true, "expiresIn": "never" }, "outcome": "success" } ``` ### Network Request Denied ```json { "timestamp": "2025-11-19T10:17:22.345Z", "type": "network_request_denied", "severity": "warning", "context": { "mcpName": "Worker Code", "sessionId": "session-1732008930123-abc123" }, "details": { "url": "http://evil.com/exfiltrate", "reason": "Domain not in whitelist" }, "outcome": "blocked" } ``` ### Security Violation ```json { "timestamp": "2025-11-19T10:18:45.678Z", "type": "security_violation", "severity": "critical", "context": { "mcpName": "code-mode", "sessionId": "session-1732008930123-abc123" }, "details": { "violation": "Prototype pollution attempt", "code": "Object.prototype.isAdmin = true;", "blocked": true }, "outcome": "blocked" } ``` ## Querying Audit Logs ### Using `grep` ```bash # Find all denied network requests grep '"type":"network_request_denied"' ~/.ncp/audit/*.jsonl # Find all critical events grep '"severity":"critical"' ~/.ncp/audit/*.jsonl # Find events from specific MCP grep '"mcpName":"lg-remote"' ~/.ncp/audit/*.jsonl ``` ### Using `jq` ```bash # Count events by type cat ~/.ncp/audit/*.jsonl | jq -s 'group_by(.type) | map({type: .[0].type, count: length})' # Find slow code executions (>1s) cat ~/.ncp/audit/*.jsonl | jq 'select(.type == "code_execution_success" and .details.durationMs > 1000)' # List all blocked network requests cat ~/.ncp/audit/*.jsonl | jq 'select(.outcome == "blocked")' ``` ### Using Node.js ```javascript import { readFile } from 'fs/promises'; // Parse audit log const content = await readFile('~/.ncp/audit/audit-2025-11-19.jsonl', 'utf-8'); const events = content.trim().split('\n').map(line => JSON.parse(line)); // Analyze events const byType = events.reduce((acc, event) => { acc[event.type] = (acc[event.type] || 0) + 1; return acc; }, {}); console.log('Events by type:', byType); // Find security violations const violations = events.filter(e => e.type === 'security_violation'); console.log(`Found ${violations.length} security violations`); ``` ## Security & Privacy ### Sensitive Data Redaction By default, audit logger redacts sensitive data: **Redacted Fields**: - `password` - `token` - `apiKey` - `secret` - `credential` - `authorization` **URL Query Parameters**: Redacted if they might contain secrets **Example**: ```json { "url": "https://api.example.com/users?api_key=<redacted>" } ``` ### Code Snippet Truncation Code snippets are truncated to 500 characters by default to prevent log bloat. **Original**: ```typescript const data = /* ... 10,000 lines of data ... */ ``` **Logged**: ```json { "codeSnippet": "const data = /* ... (truncated)", "codeLength": 50000 } ``` ## Compliance ### SOC 2 / ISO 27001 Audit logging supports compliance requirements: ✅ **Access Control**: Track who accessed what ✅ **Change Management**: Log all code executions ✅ **Incident Response**: Comprehensive event trail ✅ **Security Monitoring**: Real-time threat detection ### GDPR ✅ **Data Minimization**: Only necessary data logged ✅ **Purpose Limitation**: Logs used only for security ✅ **Storage Limitation**: Automatic log rotation ✅ **Data Protection**: Sensitive data redacted ## Best Practices ### For Users 1. **Review logs regularly**: Check for unexpected activity 2. **Monitor permissions**: Track network permission grants 3. **Archive old logs**: Keep long-term audit trail 4. **Set up alerts**: Detect security violations quickly ### For Administrators 1. **Centralize logs**: Ship to SIEM (Splunk, ELK, etc.) 2. **Set retention policy**: Balance storage vs compliance 3. **Monitor log volume**: Detect anomalies 4. **Regular audits**: Review security events monthly ### For Developers 1. **Include context**: Pass `mcpName` and `bindingName` 2. **Log failures**: Always log denied/blocked events 3. **Avoid sensitive data**: Don't log credentials directly 4. **Test audit trail**: Verify events are logged correctly ## Integration with Analytics Audit logs can be imported into NCP's analytics tool: ```bash # Import audit logs ncp analytics import --audit ~/.ncp/audit/*.jsonl # View dashboard ncp analytics dashboard ``` ## File Rotation Audit logs rotate automatically: - **Daily rotation**: New file each day (`audit-YYYY-MM-DD.jsonl`) - **Size-based rotation**: When file exceeds `maxFileSizeMB` - **Retention**: Keep last `maxFiles` files - **Compression**: Old files can be gzipped ## Performance Audit logging is designed to be low-overhead: - **Async writes**: No blocking - **Batch writes**: Multiple events buffered - **Size limits**: Prevents log bloat - **Sampling**: Can sample high-volume events **Typical overhead**: <1ms per event ## Troubleshooting ### Audit logs not created 1. Check if audit logging is enabled: ```bash echo $NCP_AUDIT_ENABLED ``` 2. Check directory permissions: ```bash ls -la ~/.ncp/audit/ ``` 3. Check for errors in NCP logs: ```bash tail -f ~/.ncp/logs/debug.log | grep audit ``` ### Disk space issues 1. Check log size: ```bash du -sh ~/.ncp/audit/ ``` 2. Reduce retention: ```bash export NCP_AUDIT_MAX_FILES=2 ``` 3. Enable compression: ```bash gzip ~/.ncp/audit/audit-*.jsonl ``` ## Roadmap Future enhancements: - [ ] Real-time streaming to SIEM - [ ] Built-in log viewer UI - [ ] Anomaly detection - [ ] Compliance report generator - [ ] Audit log encryption - [ ] Structured query language for logs ## Summary Phase 5: Monitoring & Audit provides: ✅ **Comprehensive logging**: All security events tracked ✅ **JSONL format**: Easy to parse and analyze ✅ **Sensitive data protection**: Auto-redaction ✅ **Enterprise ready**: SOC 2, ISO 27001, GDPR compliant ✅ **Low overhead**: Async, non-blocking ✅ **Actionable insights**: Query, analyze, alert **Result**: Complete visibility into Code-Mode security events for compliance, monitoring, and incident response.