# askme-server-e2e CLAUDE.md
## End-to-End Testing Strategy for MCP Servers
This directory contains comprehensive E2E tests for the Ask-Me MCP server via stdio transport. The test suite validates MCP protocol compliance, tool functionality, prompt generation, and browser bridge integration.
## Test Architecture
### Core Test Components
#### 1. MCPTestClient (`src/support/mcp-test-client.ts`)
A complete stdio MCP client implementation for testing:
- **JSON-RPC Communication**: Handles stdio transport with line-delimited JSON messages
- **MCP Protocol Support**: Initialize, list tools/prompts, call tools, get prompts
- **Process Management**: Spawns and manages server processes with proper cleanup
- **Error Handling**: Timeout management, graceful disconnection, error propagation
```typescript
// Usage example
const client = await createMCPTestClient(serverPath);
await client.initialize();
const tools = await client.listTools();
const response = await client.callTool('ask-one-question', { question: 'Test?' });
// Usage with custom port
const clientWithPort = await createMCPTestClient(serverPath, 4711);
```
#### 2. BrowserBridgeMock (`src/support/browser-bridge-mock.ts`)
Mock browser implementation for human-in-the-loop testing:
- **SSE Connection**: Simulates real browser EventSource connection
- **Auto-Response**: Configurable automatic responses to human requests
- **Request Interception**: Captures and responds to server requests
- **Response Simulation**: Generates realistic human responses for all tool types
```typescript
// Usage example
const browserMock = await createBrowserBridgeMock();
browserMock.setAutoRespond(true);
browserMock.prepareMockResponse(requestId, { answer: 'Mock response' });
```
#### 3. Protocol Helpers (`src/support/protocol-helpers.ts`)
Validation utilities for MCP protocol compliance:
- **JSON-RPC Validation**: Message format and structure validation
- **Schema Validation**: Tool and prompt schema compliance
- **Response Validation**: Content format and type checking
- **Argument Builders**: Type-safe argument construction for tools/prompts
## Test Suites
### 1. MCP Protocol Compliance Tests
Tests fundamental MCP protocol implementation:
```typescript
// Example tests
test('should initialize with proper handshake', async () => {
const processInfo = client.getProcessInfo();
expect(processInfo.connected).toBe(true);
});
test('should list available tools', async () => {
const response = await client.listTools();
validateToolsListResponse(response);
expect(response.tools.map(t => t.name)).toContain('ask-one-question');
});
```
**Coverage:**
- Server initialization and handshake
- Tool and prompt discovery
- Error handling for invalid requests
- JSON-RPC message format validation
### 2. Tool Integration Tests
End-to-end testing of human-in-the-loop tools:
```typescript
// Example tool test
test('ask-one-question tool should work end-to-end', async () => {
const requestPromise = client.callTool('ask-one-question', { question: 'Test?' });
const requests = await browserMock.waitForRequests(1);
browserMock.prepareMockResponse(requests[0].id, { answer: 'Mock answer' });
const response = await requestPromise;
validateToolCallResponse(response);
});
```
**Coverage:**
- `ask-one-question`: Single question with text response
- `ask-multiple-choice`: Multiple questions with option selection
- `challenge-hypothesis`: Agreement-based hypothesis evaluation
- `choose-next`: Option selection with structured choices
### 3. Prompt Generation Tests
Validates MCP prompt functionality:
```typescript
// Example prompt test
test('human-decision prompt should generate properly', async () => {
const args = createHumanDecisionPromptArgs('Deploy to production?');
const response = await client.getPrompt('human-decision', args);
validatePromptGetResponse(response);
});
```
**Coverage:**
- `human-decision`: Decision-making prompts
- `expert-consultation`: Domain expertise requests
- `creative-brainstorm`: Ideation and creativity prompts
- `suggest-follow-up-questions`: Conversation flow guidance
### 4. Browser Bridge Integration Tests
Tests HTTP/SSE communication between server and UI:
**Coverage:**
- SSE endpoint availability detection
- Concurrent request handling
- Request cancellation and timeout handling
- UI connection status monitoring
### 5. Error Handling and Edge Cases
Comprehensive error scenario testing:
**Coverage:**
- Malformed tool arguments
- Unknown tools and prompts
- Protocol violations
- Server restart scenarios
- Concurrent request management
## Running E2E Tests
### Prerequisites
```bash
# Build the server first
npx nx build askme-server
# Ensure no port conflicts (default port 3000 must be available)
lsof -ti :3000 | xargs kill -9 2>/dev/null || true
# Or use a custom port to avoid conflicts
# Tests can specify custom ports: --port 4711, --port 8080, etc.
```
### Test Execution
```bash
# Run all E2E tests
npx nx e2e askme-server-e2e
# Run specific test suites
npx nx e2e askme-server-e2e --testNamePattern="MCP Protocol Compliance"
npx nx e2e askme-server-e2e --testNamePattern="Tool Integration Tests"
# Run basic tests (no browser integration required)
npx nx e2e askme-server-e2e --testNamePattern="MCP Server Startup"
```
### Test Configuration
- **Timeout**: 30-60 seconds for complex integration tests
- **Dependencies**: Tests depend on `askme-server:build` and `askme-server:serve`
- **Isolation**: Each test spawns its own server process
- **Cleanup**: Automatic process cleanup in `afterAll` hooks
## Test Development Guidelines
### Adding New Tool Tests
1. Create argument builders in `protocol-helpers.ts`
2. Add validation functions for response format
3. Create end-to-end test with browser mock
4. Test both success and error scenarios
```typescript
// Example new tool test structure
test('new-tool should work end-to-end', async () => {
const args = createNewToolArgs('test input');
const requestPromise = client.callTool('new-tool', args);
const requests = await browserMock.waitForRequests(1);
browserMock.prepareMockResponse(requests[0].id, mockResponse);
const response = await requestPromise;
validateToolCallResponse(response);
expect(response.content).toContain(expectedContent);
});
```
### Adding New Prompt Tests
1. Create prompt argument builders
2. Add response validation
3. Test parameter validation and error cases
```typescript
// Example new prompt test
test('new-prompt should generate properly', async () => {
const args = createNewPromptArgs('test context');
const response = await client.getPrompt('new-prompt', args);
validatePromptGetResponse(response);
expect(response.messages[0].content.text).toContain('expected content');
});
```
## Common Issues and Solutions
### Port Conflicts
**Issue**: `EADDRINUSE: address already in use :::3000`
**Solution**: Kill processes using port 3000 before running tests
```bash
lsof -ti :3000 | xargs kill -9 2>/dev/null
```
### Test Timeouts
**Issue**: Tests timeout during server startup
**Solution**:
- Ensure server is built: `npx nx build askme-server`
- Check server logs for startup errors
- Increase timeout for complex integration tests
### Browser Bridge Failures
**Issue**: Browser mock connection failures
**Solution**:
- Use test-specific ports or skip browser-dependent tests
- Implement proper retry logic for connection establishment
- Mock browser bridge for protocol-only testing
## Debugging E2E Tests
### Server Logs
```typescript
// Enable server logging in tests
serverProcess.stderr.on('data', (data) => {
console.log('Server:', data.toString());
});
```
### MCP Message Tracing
```typescript
// Log JSON-RPC messages
client.on('message', (message) => {
console.log('MCP Message:', JSON.stringify(message, null, 2));
});
```
### Process Management
```typescript
// Check process status
const info = client.getProcessInfo();
console.log('Server PID:', info.pid, 'Connected:', info.connected);
```
## Test Maintenance
### Updating for New MCP Features
1. Update `MCPTestClient` for new protocol features
2. Add validation helpers for new message types
3. Create test cases for new capabilities
4. Update error code mappings
### Performance Considerations
- Tests spawn real server processes - expect 10-30 second execution times
- Use process pooling for faster test runs if needed
- Mock browser bridge for protocol-only testing
- Clean up processes properly to avoid resource leaks
## Integration with CI/CD
### Build Pipeline Integration
```yaml
# Example GitHub Actions step
- name: Run E2E Tests
run: |
npx nx build askme-server
npx nx e2e askme-server-e2e
```
### Environment Requirements
- Node.js with stdio support
- Available ports (3000 for browser bridge)
- Sufficient memory for multiple Node processes
- Proper process cleanup capabilities
