README_TESTING.md•8.87 kB
# Testing Guide for API Aggregator MCP Server
This guide covers testing best practices and how to run tests for the API Aggregator MCP Server.
## Table of Contents
1. [Test Structure](#test-structure)
2. [Running Tests](#running-tests)
3. [Writing Tests](#writing-tests)
4. [Test Categories](#test-categories)
5. [Coverage](#coverage)
6. [Continuous Integration](#continuous-integration)
## Test Structure
Our test suite follows Python testing best practices:
```
tests/
├── __init__.py # Makes tests a package
├── conftest.py # Shared fixtures and configuration
├── test_config.py # Configuration tests
├── test_errors.py # Error handling tests
├── test_weather.py # Weather tool tests
├── test_server.py # MCP server tests
└── test_main.py # Main application tests
```
### Key Testing Principles
1. **AAA Pattern**: Arrange, Act, Assert
2. **Test Isolation**: Each test is independent
3. **Descriptive Names**: Test names explain what's being tested
4. **Mock External Dependencies**: Don't make real API calls in tests
## Running Tests
### Prerequisites
Install development dependencies:
```bash
pip install -r requirements-dev.txt
```
### Basic Test Execution
Run all tests:
```bash
pytest
```
Run tests with verbose output:
```bash
pytest -v
```
Run a specific test file:
```bash
pytest tests/test_weather.py
```
Run a specific test class:
```bash
pytest tests/test_weather.py::TestWeatherService
```
Run a specific test method:
```bash
pytest tests/test_weather.py::TestWeatherService::test_get_current_weather_success
```
### Test Filtering
Run only unit tests:
```bash
pytest -m unit
```
Run only integration tests:
```bash
pytest -m integration
```
Skip slow tests:
```bash
pytest -m "not slow"
```
### Parallel Execution
For faster test execution with pytest-xdist:
```bash
pip install pytest-xdist
pytest -n auto # Use all CPU cores
```
## Writing Tests
### Test File Naming
- Test files: `test_*.py`
- Test classes: `Test*`
- Test functions: `test_*`
### Basic Test Structure
```python
"""Tests for my_module.py"""
import pytest
from unittest.mock import Mock, patch, AsyncMock
from src.my_module import MyClass
class TestMyClass:
"""Test cases for MyClass."""
def test_my_method_success(self):
"""Test successful execution of my_method."""
# Arrange
instance = MyClass()
# Act
result = instance.my_method("input")
# Assert
assert result == "expected_output"
@pytest.mark.asyncio
async def test_async_method(self):
"""Test async method execution."""
instance = MyClass()
result = await instance.async_method()
assert result is not None
```
### Mocking Guidelines
#### Mock External APIs
```python
@patch("src.tools.weather.aiohttp.ClientSession")
async def test_weather_api_call(self, mock_session):
mock_response = AsyncMock()
mock_response.status = 200
mock_response.json.return_value = {"temp": 25}
mock_session.return_value.get.return_value.__aenter__ = AsyncMock(return_value=mock_response)
# Test code here
```
#### Mock Configuration
```python
@patch("src.utils.config.get_settings")
def test_with_mock_settings(self, mock_get_settings):
mock_settings = Mock()
mock_settings.api_key = "test_key"
mock_get_settings.return_value = mock_settings
# Test code here
```
### Testing Async Code
```python
@pytest.mark.asyncio
async def test_async_function():
"""Test async function."""
result = await my_async_function()
assert result == expected_value
```
### Testing Exceptions
```python
def test_exception_handling():
"""Test that function raises expected exception."""
with pytest.raises(ValueError) as exc_info:
function_that_should_raise()
assert "expected error message" in str(exc_info.value)
```
### Using Fixtures
```python
@pytest.fixture
def sample_data():
"""Provide sample data for tests."""
return {"key": "value"}
def test_with_fixture(sample_data):
"""Test using fixture data."""
assert sample_data["key"] == "value"
```
## Test Categories
We use pytest markers to categorize tests:
### Unit Tests
```python
@pytest.mark.unit
def test_calculation():
"""Test pure function with no dependencies."""
pass
```
### Integration Tests
```python
@pytest.mark.integration
def test_api_integration():
"""Test integration between components."""
pass
```
### Slow Tests
```python
@pytest.mark.slow
def test_performance():
"""Test that takes significant time."""
pass
```
### External API Tests
```python
@pytest.mark.external_api
def test_real_api():
"""Test that hits real external APIs (use sparingly)."""
pass
```
## Coverage
### Running with Coverage
Install pytest-cov:
```bash
pip install pytest-cov
```
Run tests with coverage:
```bash
pytest --cov=src --cov-report=html --cov-report=term-missing
```
### Coverage Configuration
Uncomment in `pytest.ini` to enable coverage by default:
```ini
addopts = --cov=src --cov-report=html --cov-report=term-missing --cov-fail-under=80
```
### Coverage Reports
- **Terminal**: Shows missing lines
- **HTML**: Detailed report in `htmlcov/index.html`
### Coverage Targets
- **Unit tests**: Aim for 90%+ coverage
- **Integration tests**: Focus on critical paths
- **Overall**: Maintain 80%+ coverage
## Best Practices
### 1. Test Naming
```python
# Good
def test_get_weather_returns_normalized_data_when_api_succeeds():
pass
# Bad
def test_weather():
pass
```
### 2. Test Organization
```python
class TestWeatherService:
"""Group related tests together."""
def test_success_case(self):
pass
def test_error_case(self):
pass
def test_edge_case(self):
pass
```
### 3. Mock at the Right Level
```python
# Good - Mock external dependency
@patch("src.tools.weather.aiohttp.ClientSession")
def test_weather_service(self, mock_session):
pass
# Bad - Mock internal function you're testing
@patch("src.tools.weather.WeatherService.get_current_weather")
def test_weather_service(self, mock_method):
pass
```
### 4. Use Realistic Test Data
```python
# Good - Realistic data structure
sample_weather_response = {
"coord": {"lon": -122.42, "lat": 37.77},
"weather": [{"main": "Clouds", "description": "few clouds"}],
"main": {"temp": 18.5, "humidity": 72}
}
# Bad - Minimal or unrealistic data
sample_response = {"temp": 20}
```
### 5. Test Error Conditions
```python
def test_api_error_handling():
"""Test various error conditions."""
# Test missing API key
# Test network errors
# Test invalid responses
# Test rate limiting
```
## Common Patterns
### Testing FastAPI Endpoints
```python
from fastapi.testclient import TestClient
def test_api_endpoint(mcp_server):
client = TestClient(mcp_server.app)
response = client.get("/tools")
assert response.status_code == 200
assert response.json() == []
```
### Testing Async Handlers
```python
@pytest.mark.asyncio
async def test_weather_handler():
parameters = {"city": "San Francisco"}
with patch("src.tools.weather.weather_service") as mock_service:
mock_service.get_current_weather = AsyncMock(return_value={"temp": 20})
result = await get_weather_handler(parameters)
assert result["temp"] == 20
```
### Testing Configuration
```python
def test_settings_from_environment():
with patch.dict(os.environ, {"API_KEY": "test_key"}):
settings = Settings()
assert settings.api_key == "test_key"
```
## Troubleshooting
### Common Issues
1. **Async test not running**: Add `@pytest.mark.asyncio`
2. **Mock not working**: Check import paths and patch locations
3. **Test isolation**: Ensure tests don't share state
4. **Fixture scope**: Use appropriate fixture scope for your needs
### Debugging Tests
```bash
# Run with debugging
pytest --pdb
# Run with print statements
pytest -s
# Run specific test with verbose output
pytest -v tests/test_weather.py::test_specific_method
```
## Continuous Integration
For CI/CD pipelines, use:
```bash
# Install dependencies
pip install -r requirements-dev.txt
# Run linting
flake8 src tests
black --check src tests
# Run type checking
mypy src
# Run tests with coverage
pytest --cov=src --cov-report=xml --cov-fail-under=80
# Generate coverage reports
pytest --cov=src --cov-report=html
```
## Additional Resources
- [pytest documentation](https://docs.pytest.org/)
- [pytest-asyncio documentation](https://pytest-asyncio.readthedocs.io/)
- [unittest.mock documentation](https://docs.python.org/3/library/unittest.mock.html)
- [FastAPI testing](https://fastapi.tiangolo.com/tutorial/testing/)