Curupira

# WebSocket Verification Issues with Chrome DevTools Protocol - Research Report ## Executive Summary This research analyzes WebSocket verification issues in the Curupira CDP implementation, specifically focusing on the `Target.getVersion` timing problems after WebSocket connection. The current implementation shows verification failures despite successful WebSocket connections, particularly with Browserless environments. ## Current Issue Analysis ### Location of Issue - **File**: `/mcp-server/src/chrome/client.ts` - **Lines**: 139-156 - **Method**: `connectToBrowser()` ### Current Implementation ```typescript this.browserWs.on('open', () => { clearTimeout(timeout); this.logger.info('Browser WebSocket connected successfully'); this.logger.debug({ readyState: this.browserWs?.readyState }, 'WebSocket ready state'); // Ensure WebSocket is truly ready by testing a simple command this.sendBrowserCommand('Target.getVersion') .then((version) => { this.logger.info({ version }, 'Browser WebSocket verified with version check'); resolve(); }) .catch((error) => { this.logger.error({ error }, 'Browser WebSocket verification failed'); if (this.browserWs) { this.browserWs.close(); } reject(new Error('Browser WebSocket verification failed')); }); }); ``` ### Problems Identified 1. **Immediate Command Execution**: `Target.getVersion` is sent immediately after the 'open' event 2. **No CDP Initialization Wait**: The CDP protocol may not be fully initialized when the WebSocket opens 3. **Missing Ready State Verification**: No explicit check for CDP protocol readiness 4. **Browserless Specific Timing**: Browserless environments may have additional initialization delays ## Industry Best Practices ### 1. Puppeteer Approach - Uses internal connection management with message queue - Implements promise-based command handling with unique IDs - Monitors all CDP traffic with DEBUG environment variable - Handles disconnections gracefully with event-based cleanup ### 2. Chrome-Remote-Interface Patterns - **Event-based verification**: Waits for 'connect' event before operations - **Promise-based connection**: Returns fulfilled promise only after full initialization - **Ready event**: Some implementations wait for a 'ready' event after connect - **Domain enabling**: Enables necessary CDP domains before use ### 3. CDP Protocol Considerations - **Message Order**: CDP guarantees message order for certain operations - **Session Management**: Root browser session created on WebSocket connect - **Target Attachment**: Must use `Target.getTargets` before `Target.attachToTarget` - **Implicit Timing**: Some CDP operations have implicit timing guarantees ## Common WebSocket Verification Patterns ### Pattern 1: Delay After Open ```javascript ws.on('open', async () => { // Add small delay for CDP initialization await new Promise(resolve => setTimeout(resolve, 100)); // Then verify connection }); ``` ### Pattern 2: Retry with Backoff ```javascript async function verifyConnection(retries = 3) { for (let i = 0; i < retries; i++) { try { const version = await sendCommand('Target.getVersion'); return version; } catch (error) { if (i === retries - 1) throw error; await new Promise(r => setTimeout(r, (i + 1) * 100)); } } } ``` ### Pattern 3: Protocol Handshake ```javascript // Wait for first CDP event as confirmation const ready = new Promise((resolve) => { ws.once('message', (data) => { const msg = JSON.parse(data); if (msg.method || msg.id) { resolve(); } }); }); ``` ### Pattern 4: Enable Domain First ```javascript ws.on('open', async () => { // Enable Target domain first await sendCommand('Target.enable'); // Then get version const version = await sendCommand('Target.getVersion'); }); ``` ## Browserless Specific Considerations 1. **Connection URL Patterns**: Browserless uses standard WebSocket patterns but may have additional overhead 2. **Session Persistence**: Uses `Browserless.reconnect` CDP command for session management 3. **Timeout Configuration**: Default no timeout, relies on browser lifecycle 4. **Flattened Targets**: Uses `flatten: true` when attaching to targets ## Recommended Solutions ### Solution 1: Add Initialization Delay (Quick Fix) ```typescript this.browserWs.on('open', async () => { clearTimeout(timeout); this.logger.info('Browser WebSocket connected successfully'); // Add small delay for CDP initialization await new Promise(resolve => setTimeout(resolve, 100)); // Then verify with Target.getVersion try { const version = await this.sendBrowserCommand('Target.getVersion'); this.logger.info({ version }, 'Browser WebSocket verified with version check'); resolve(); } catch (error) { this.logger.error({ error }, 'Browser WebSocket verification failed'); if (this.browserWs) { this.browserWs.close(); } reject(new Error('Browser WebSocket verification failed')); } }); ``` ### Solution 2: Implement Retry Logic (Robust) ```typescript private async verifyBrowserConnection(maxRetries = 3): Promise<void> { for (let attempt = 0; attempt < maxRetries; attempt++) { try { const version = await this.sendBrowserCommand('Target.getVersion'); this.logger.info({ version, attempt }, 'Browser WebSocket verified'); return; } catch (error) { this.logger.warn({ error, attempt }, 'Verification attempt failed'); if (attempt === maxRetries - 1) { throw error; } // Exponential backoff await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 100)); } } } ``` ### Solution 3: Wait for CDP Event (Most Reliable) ```typescript this.browserWs.on('open', () => { clearTimeout(timeout); this.logger.info('Browser WebSocket connected, waiting for CDP ready...'); // Set up one-time listener for first CDP message const onFirstMessage = (data: string) => { try { const msg = JSON.parse(data); if (msg.method || msg.id) { this.browserWs.removeListener('message', onFirstMessage); // Now safe to verify this.sendBrowserCommand('Target.getVersion') .then((version) => { this.logger.info({ version }, 'Browser WebSocket verified'); resolve(); }) .catch((error) => { this.logger.error({ error }, 'Verification failed'); reject(error); }); } } catch (e) { // Ignore parse errors } }; this.browserWs.on('message', onFirstMessage); }); ``` ### Solution 4: Enable Target Domain First (CDP Compliant) ```typescript this.browserWs.on('open', async () => { clearTimeout(timeout); this.logger.info('Browser WebSocket connected successfully'); try { // Enable Target domain first await this.sendBrowserCommand('Target.enable'); // Small delay for domain initialization await new Promise(resolve => setTimeout(resolve, 50)); // Now get version const version = await this.sendBrowserCommand('Target.getVersion'); this.logger.info({ version }, 'Browser WebSocket verified with version check'); resolve(); } catch (error) { this.logger.error({ error }, 'Browser WebSocket verification failed'); if (this.browserWs) { this.browserWs.close(); } reject(new Error('Browser WebSocket verification failed')); } }); ``` ## Testing Recommendations 1. **Unit Tests**: Mock WebSocket with varying initialization delays 2. **Integration Tests**: Test against real Browserless instances 3. **Timing Tests**: Measure time between 'open' event and successful CDP command 4. **Error Scenarios**: Test with slow/unreliable connections 5. **Load Tests**: Verify behavior under concurrent connections ## Performance Considerations - **Initialization Delay**: 50-100ms is typically sufficient - **Retry Overhead**: Exponential backoff prevents overwhelming the server - **Connection Pool**: Consider connection pooling for multiple tabs - **Keep-Alive**: Implement ping/pong for long-lived connections ## Conclusion The current implementation's immediate execution of `Target.getVersion` after WebSocket 'open' event is likely too aggressive for Browserless environments. The recommended approach is to implement Solution 2 (Retry Logic) or Solution 3 (Wait for CDP Event) for maximum reliability. These solutions have minimal performance impact while significantly improving connection stability. ## References 1. [Chrome DevTools Protocol Documentation](https://chromedevtools.github.io/devtools-protocol/) 2. [Puppeteer Connection Implementation](https://github.com/puppeteer/puppeteer/blob/main/src/common/Connection.ts) 3. [Chrome-Remote-Interface](https://github.com/cyrus-and/chrome-remote-interface) 4. [Browserless Documentation](https://docs.browserless.io/) 5. [WebSocket MDN Documentation](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)