mcp-adr-analysis-server

# 🧪 MCP ADR Analysis Server Testing Guide **Comprehensive testing guide for developers and contributors** > **Version**: 2.1.0 | **Test Files**: 70+ | **Coverage**: 100% (1,739 tests passing) --- ## 📖 Table of Contents 1. [Testing Overview](#testing-overview) 2. [Test Suite Structure](#test-suite-structure) 3. [Running Tests](#running-tests) 4. [Test Categories](#test-categories) 5. [Writing New Tests](#writing-new-tests) 6. [CI/CD Integration](#cicd-integration) 7. [Troubleshooting Tests](#troubleshooting-tests) 8. [Performance Testing](#performance-testing) --- ## 🎯 Testing Overview The MCP ADR Analysis Server has a comprehensive test suite with **1,739 passing tests** across **70+ test files**, achieving **100% test success rate**. Our testing strategy follows methodological pragmatism principles with systematic verification and explicit acknowledgment of test coverage. ### Test Statistics - **Total Tests**: 1,739 passing - **Test Suites**: 66 suites - **Test Files**: 70+ files - **Coverage**: >80% target achieved - **Execution Time**: ~34 seconds - **Success Rate**: 100% ### Testing Philosophy - **Methodological Pragmatism**: Evidence-based testing with confidence scoring - **Integration-Focused**: Real-world scenario testing over unit isolation - **Quality Gates**: Zero-tolerance for failing tests in production - **Continuous Validation**: Automated testing in CI/CD pipeline --- ## 📁 Test Suite Structure ### Core Test Categories ``` tests/ ├── 📋 Core Framework Tests │ ├── ape.test.ts # APE framework testing │ ├── cache.test.ts # Caching system tests │ ├── config.test.ts # Configuration validation │ ├── index.test.ts # Main server tests │ ├── knowledge-generation.test.ts # AI knowledge generation │ ├── mcp-server.test.ts # MCP protocol tests │ ├── reflexion.test.ts # Reflexion framework │ └── types.test.ts # Type system validation │ ├── 🛠️ Tool Tests (18 core tools) │ ├── adr-bootstrap-validation-tool.test.ts │ ├── adr-suggestion-tool.test.ts │ ├── content-masking-tool.test.ts │ ├── deployment-analysis-tool.test.ts │ ├── deployment-guidance-tool.test.ts │ ├── deployment-readiness-tool.test.ts │ ├── environment-analysis-tool.test.ts │ ├── interactive-adr-planning-tool.test.ts │ ├── mcp-planning-tool.test.ts │ ├── memory-loading-tool.test.ts │ ├── research-integration-tool.test.ts │ ├── review-existing-adrs-tool.test.ts │ ├── rule-generation-tool.test.ts │ ├── smart-git-push-tool-v2.test.ts │ ├── smart-git-push-tool.test.ts │ ├── tool-chain-orchestrator.test.ts │ └── troubleshoot-guided-workflow-tool.test.ts │ ├── 🔧 Utility Tests │ ├── ai-executor.test.ts │ ├── deployment-analysis.test.ts │ ├── gitleaks-detector.test.ts │ ├── interactive-approval.test.ts │ ├── llm-artifact-detector.test.ts │ ├── location-filter.test.ts │ ├── mcp-response-validator.test.ts │ ├── memory-entity-manager.test.ts │ ├── memory-relationship-mapper.test.ts │ ├── memory-rollback-manager.test.ts │ ├── memory-transformation.test.ts │ ├── output-masking.test.ts │ ├── prompt-composition.test.ts │ ├── prompt-execution.test.ts │ ├── release-readiness-detector.test.ts │ ├── research-documentation.test.ts │ ├── research-integration.test.ts │ ├── research-questions.test.ts │ ├── rule-format.test.ts │ ├── rule-generation.test.ts │ ├── test-helpers.test.ts │ ├── test-infrastructure.test.ts │ ├── todo-file-watcher.test.ts │ └── tree-sitter-analyzer.test.ts │ ├── 🎯 Prompt Tests │ ├── deployment-analysis-prompts.test.ts │ ├── environment-analysis-prompts.test.ts │ ├── research-integration-prompts.test.ts │ ├── research-question-prompts.test.ts │ ├── rule-generation-prompts.test.ts │ └── security-prompts.test.ts │ ├── 🏗️ Integration Tests │ ├── memory-migration-integration.test.ts │ ├── memory-migration-simple.test.ts │ └── memory-system-integration.test.ts │ ├── ⚡ Performance Tests │ └── memory-performance.test.ts │ └── 📊 Configuration Tests └── ai-config.test.ts ``` --- ## 🚀 Running Tests ### Basic Test Commands ```bash # Run all tests npm test # Run tests with coverage npm run test:coverage # Run tests in watch mode (development) npm run test:watch # Run specific test file npm test -- tests/tools/adr-suggestion-tool.test.ts # Run tests matching pattern npm test -- --testNamePattern="memory" # Run tests with verbose output npm test -- --verbose ``` ### Advanced Test Commands ```bash # Run integration tests only npm run test:integration # Run unit tests only npm run test:unit # Run performance tests npm run test:performance # Run advanced prompting tests npm run test:advanced # Run smoke tests (quick validation) npm test -- tests/smoke.test.ts ``` ### Make Commands ```bash # Run all tests (same as npm test) make test # Run tests with coverage report make test-coverage # Run lint and tests together make check # Clean and test make clean test ``` --- ## 🎨 Test Categories ### 1. Unit Tests **Purpose**: Test individual functions and components in isolation ```typescript // Example: Function unit test describe('generateAdrFromDecision', () => { it('should generate ADR with proper structure', async () => { const decision = { title: 'Use React for Frontend', context: 'Need modern UI framework', decision: 'Adopt React 18', }; const result = await generateAdrFromDecision(decision); expect(result.title).toBe('Use React for Frontend'); expect(result.status).toBe('proposed'); expect(result.content).toContain('## Context'); }); }); ``` ### 2. Integration Tests **Purpose**: Test tool interactions and workflows ```typescript // Example: Integration test describe('Memory System Integration', () => { it('should load ADRs and query entities', async () => { // Load ADRs into memory const loadResult = await memoryTool.execute({ action: 'load_adrs', forceReload: true, }); expect(loadResult.isError).toBeFalsy(); // Query loaded entities const queryResult = await memoryTool.execute({ action: 'query_entities', query: { entityTypes: ['architectural_decision'] }, }); expect(queryResult.entities.length).toBeGreaterThan(0); }); }); ``` ### 3. Tool Tests **Purpose**: Test MCP tool functionality and parameters ```typescript // Example: Tool test describe('analyze_project_ecosystem', () => { it('should analyze project with enhanced mode', async () => { const result = await analyzeProjectEcosystem({ projectPath: '/test/project', enhancedMode: true, knowledgeEnhancement: true, analysisDepth: 'comprehensive', }); expect(result.content).toBeDefined(); expect(result.confidence).toBeGreaterThan(0.8); expect(result.technologies).toContain('typescript'); }); }); ``` ### 4. Prompt Tests **Purpose**: Validate prompt generation and AI integration ```typescript // Example: Prompt test describe('generateAdrSuggestionPrompt', () => { it('should generate comprehensive prompt', () => { const prompt = generateAdrSuggestionPrompt({ projectContext: 'React application', technologies: ['react', 'typescript'], analysisResults: mockAnalysis, }); expect(prompt).toContain('React application'); expect(prompt).toContain('typescript'); expect(prompt.length).toBeGreaterThan(100); }); }); ``` ### 5. Performance Tests **Purpose**: Validate system performance and resource usage ```typescript // Example: Performance test describe('Memory Performance', () => { it('should handle large entity sets efficiently', async () => { const startTime = Date.now(); const entities = generateLargeEntitySet(1000); await memoryManager.batchUpsertEntities(entities); const duration = Date.now() - startTime; expect(duration).toBeLessThan(5000); // 5 seconds max }); }); ``` --- ## ✍️ Writing New Tests ### Test File Template ```typescript /** * Test file for [COMPONENT_NAME] * * Tests cover: * - Basic functionality * - Edge cases * - Error handling * - Integration scenarios */ import { describe, it, expect, beforeEach, afterEach } from '@jest/globals'; import { [COMPONENT_NAME] } from '../src/path/to/component.js'; describe('[COMPONENT_NAME]', () => { beforeEach(() => { // Setup before each test }); afterEach(() => { // Cleanup after each test }); describe('basic functionality', () => { it('should perform expected operation', () => { // Test implementation expect(true).toBe(true); }); }); describe('edge cases', () => { it('should handle empty input', () => { // Edge case testing }); }); describe('error handling', () => { it('should throw on invalid input', () => { expect(() => { // Error condition }).toThrow(); }); }); }); ``` ### Testing Best Practices #### 1. Test Structure (AAA Pattern) ```typescript it('should describe what it tests', () => { // Arrange - Set up test data const input = { test: 'data' }; // Act - Execute the function const result = functionUnderTest(input); // Assert - Verify the result expect(result).toBe(expectedValue); }); ``` #### 2. Descriptive Test Names ```typescript // ❌ Bad it('should work', () => {}); // ✅ Good it('should generate ADR with proper MADR format when template is specified', () => {}); ``` #### 3. Test Data Management ```typescript // Use factories for complex test data const createMockAnalysis = (overrides = {}) => ({ projectPath: '/test/project', technologies: ['typescript', 'react'], confidence: 0.95, ...overrides, }); ``` #### 4. Async Testing ```typescript // Proper async test handling it('should handle async operations', async () => { const result = await asyncFunction(); expect(result).toBeDefined(); }); ``` ### Test Coverage Requirements - **Minimum Coverage**: 80% (currently exceeding this) - **Function Coverage**: All exported functions must be tested - **Branch Coverage**: All conditional branches tested - **Integration Coverage**: All tool interactions tested --- ## 🔄 CI/CD Integration ### GitHub Actions Workflow The test suite runs automatically on: - **Pull Requests**: All tests must pass - **Push to Main**: Full test suite execution - **Scheduled**: Daily test runs for regression detection ```yaml # .github/workflows/test.yml name: Test Suite on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' - run: npm ci - run: npm test - run: npm run test:coverage ``` ### Pre-commit Hooks Tests run automatically before commits: ```bash # .husky/pre-commit #!/usr/bin/env sh npm run lint npm test npm run build ``` ### Quality Gates - **Zero Failing Tests**: No commits allowed with failing tests - **Coverage Threshold**: Must maintain >80% coverage - **Performance Regression**: Tests must complete within time limits - **Security Validation**: Security-focused tests must pass --- ## 🔧 Troubleshooting Tests ### Common Test Issues #### 1. Memory/Timeout Issues ```bash # Increase Jest timeout npm test -- --testTimeout=30000 # Run tests with more memory node --max-old-space-size=4096 node_modules/.bin/jest ``` #### 2. Path Resolution Issues ```typescript // Use dynamic path resolution import { fileURLToPath } from 'url'; import { dirname, join } from 'path'; const __filename = fileURLToPath(import.meta.url); const __dirname = dirname(__filename); const testWorkspace = join(__dirname, '../../test-workspace'); ``` #### 3. Mock Issues ```typescript // Proper mock cleanup afterEach(() => { jest.clearAllMocks(); jest.resetModules(); }); ``` #### 4. Tree-sitter Parser Issues ```bash # Tree-sitter parsers may fail in test environment # Tests gracefully handle parser failures console.warn('Tree-sitter parser failed, falling back to regex analysis'); ``` ### Debugging Tests ```bash # Run single test with debugging node --inspect-brk node_modules/.bin/jest tests/specific-test.test.ts # Run with verbose logging DEBUG=* npm test # Run with Jest debugging npm test -- --detectOpenHandles --forceExit ``` --- ## ⚡ Performance Testing ### Performance Test Categories #### 1. Memory Performance ```typescript describe('Memory Performance', () => { it('should handle large datasets efficiently', async () => { const startMemory = process.memoryUsage().heapUsed; // Perform memory-intensive operation await processLargeDataset(); const endMemory = process.memoryUsage().heapUsed; const memoryIncrease = endMemory - startMemory; expect(memoryIncrease).toBeLessThan(100 * 1024 * 1024); // 100MB limit }); }); ``` #### 2. Execution Time Performance ```typescript describe('Execution Performance', () => { it('should complete analysis within time limit', async () => { const startTime = performance.now(); await analyzeProjectEcosystem({ projectPath: '/large/project' }); const duration = performance.now() - startTime; expect(duration).toBeLessThan(10000); // 10 second limit }); }); ``` #### 3. Concurrent Operation Performance ```typescript describe('Concurrency Performance', () => { it('should handle multiple simultaneous requests', async () => { const requests = Array(10) .fill() .map(() => analyzeContentSecurity({ content: 'test content' })); const results = await Promise.all(requests); expect(results).toHaveLength(10); results.forEach(result => { expect(result.isError).toBeFalsy(); }); }); }); ``` --- ## 📊 Test Metrics and Reporting ### Coverage Reports ```bash # Generate coverage report npm run test:coverage # View coverage in browser open coverage/lcov-report/index.html ``` ### Test Results Current test metrics: - **Total Tests**: 1,739 ✅ - **Test Suites**: 66 ✅ - **Success Rate**: 100% ✅ - **Execution Time**: ~34 seconds ✅ - **Coverage**: >80% target achieved ✅ ### Continuous Monitoring - **Daily Test Runs**: Automated regression detection - **Performance Tracking**: Monitor test execution times - **Coverage Tracking**: Ensure coverage doesn't decrease - **Flaky Test Detection**: Identify unstable tests --- ## 🎯 Contributing to Tests ### Adding New Tests 1. **Identify Test Category**: Unit, integration, or performance 2. **Follow Naming Convention**: `component-name.test.ts` 3. **Use Test Template**: Follow established patterns 4. **Ensure Coverage**: Test all branches and edge cases 5. **Add Documentation**: Document complex test scenarios ### Test Review Checklist - [ ] Tests follow AAA pattern (Arrange, Act, Assert) - [ ] Descriptive test names explain what is being tested - [ ] Edge cases and error conditions covered - [ ] No hardcoded paths or environment-specific code - [ ] Proper async/await handling - [ ] Mock cleanup in afterEach hooks - [ ] Performance considerations addressed --- ## 📚 Additional Resources - **[Jest Documentation](https://jestjs.io/./getting-started)** - Testing framework guide - **[Testing Best Practices](https://github.com/goldbergyoni/javascript-testing-best-practices)** - Industry standards - **[CI/CD Guide](./how-to-guides/cicd-integration.md)** - Continuous integration setup - **[Contributing Guide](./../../CONTRIBUTING.md)** - Project contribution guidelines --- **🧪 Your test suite is comprehensive and production-ready with 100% success rate!** _This testing guide follows methodological pragmatism principles with systematic verification and explicit acknowledgment of test coverage and limitations._