GitHub Repository Analyzer

Testing Strategy: GitHub Repository Analyzer MCP Server ======================================================= This document outlines a comprehensive testing approach for the GitHub Repository Analyzer MCP server, including manual testing processes, validation criteria, and troubleshooting strategies. Testing with MCP Inspector -------------------------- The MCP Inspector is a tool provided by the MCP Python SDK that allows you to test your server locally before integrating with LLM clients. ### Starting the Inspector 1. Open a terminal or command prompt 2. Navigate to your project directory 3. Run the command: ``` mcp dev server.py ``` 4. A web interface will open in your default browser ### Using the Inspector The MCP Inspector provides a simple interface to: 1. View all registered tools and resources 2. Manually invoke tools with test inputs 3. See raw responses and errors 4. Inspect the MCP protocol messages ### Test Flow for Each Tool For each tool, follow this testing process: 1. Select the tool from the list of available tools 2. Enter test parameters (e.g., a repository name) 3. Execute the tool and observe the response 4. Verify the response matches the expected format and contains valid data Test Cases for Each Tool ------------------------ ### Repository Information Tool **Test Cases:** 1. Retrieve information for a popular repository (e.g., "microsoft/vscode") 2. Retrieve information for a small/personal repository 3. Test with a non-existent repository 4. Test with an invalid repository name format **Validation Criteria:** - Response includes all expected fields (name, description, stats, etc.) - Numeric values (stars, forks, etc.) are reasonable - URLs are valid and correctly formatted - Error handling works appropriately for invalid inputs ### Issue Analysis Tool **Test Cases:** 1. List issues from a repository with many open issues 2. List issues from a repository with few or no issues 3. Test with different state filters (open, closed, all) 4. Test with different maximum issue counts **Validation Criteria:** - Response includes correct number of issues (up to the maximum) - Issue data includes all expected fields - State filtering works correctly - Pagination works for repositories with many issues ### README Access Resource **Test Cases:** 1. Access README from a repository with a standard README.md 2. Test with a repository that has a different README format (e.g., README.rst) 3. Test with a repository that has no README 4. Test with a repository that has a very large README **Validation Criteria:** - Content matches the actual README on GitHub - Formatting is preserved appropriately - Error handling works for repositories without READMEs - Large READMEs are handled correctly ### Commit History Tool **Test Cases:** 1. Retrieve commits from an active repository 2. Test with different time ranges (7 days, 30 days, etc.) 3. Test with different maximum commit counts 4. Test with a repository with no recent commits **Validation Criteria:** - Commit data includes all expected fields - Date filtering works correctly - Maximum count is respected - Results are ordered by date (newest first) ### Activity Analysis Tool **Test Cases:** 1. Analyze a highly active repository 2. Analyze a repository with moderate activity 3. Analyze a repository with no recent activity 4. Test with different time periods **Validation Criteria:** - Metrics are calculated correctly - Results match what can be observed on GitHub - Zero-activity cases are handled appropriately - Different time periods produce different results ### Visualization Tool **Test Cases:** 1. Generate a chart for a repository with consistent activity 2. Generate a chart for a repository with irregular activity 3. Generate a chart for a repository with no activity 4. Test with different time periods **Validation Criteria:** - Chart is generated without errors - Visual representation matches the actual data - Chart is readable and informative - Time period is reflected correctly in the chart Integration Testing ------------------- After individual tools have been tested, perform integration testing: 1. Test multiple tool calls in sequence 2. Verify that tools work correctly when used together 3. Check for any unexpected interactions or dependencies Performance Testing ------------------- Assess the performance of your server: 1. Test with large repositories to check for timeouts or errors 2. Monitor response times for different tools 3. Identify any bottlenecks in the implementation Error Handling Validation ------------------------- Deliberately introduce errors to verify proper handling: 1. Disconnect from the internet to test network error handling 2. Use an invalid GitHub token to test authentication errors 3. Try to exceed GitHub API rate limits to test rate limiting handling Checkpoints List ---------------- - [x] Set up and understand the MCP Inspector - [x] Create test cases for the Repository Information Tool - [x] Create test cases for the Issue Analysis Tool - [x] Create test cases for the README Access Resource - [x] Create test cases for the Commit History Tool - [x] Create test cases for the Activity Analysis Tool - [x] Create test cases for the Visualization Tool - [x] Plan integration testing approach - [x] Consider performance testing requirements - [x] Prepare error handling validation