# News Aggregator API (MCP Server)

A modular, scalable news aggregation backend built with MCP server conventions and TypeScript. This API provides a unified interface to TheNewsAPI for retrieving current and historical news articles with advanced filtering capabilities. Designed specifically for consumption by AI agents, this API prioritizes structured data and consistent patterns.

## Features

- **Unified API**: Consistent interface for accessing news data
- **TypeScript**: Fully typed for reliable development
- **MCP Architecture**: Scalable, modular design with separation of concerns
- **Multiple Endpoints**: Access various types of news content
- **Environment Variables**: Secure configuration management
- **Error Handling**: Consistent error formats with appropriate status codes
- **Filtering**: Support for various filtering parameters
- **Caching**: Intelligent caching system with management endpoints
- **Interactive Documentation**: Swagger-based API documentation
- **AI-Friendly Responses**: Structured data optimized for machine consumption

## Endpoints

### News Endpoints

| Endpoint | Description | Example |
|----------|-------------|--------|
| `/api/news/top` | Get top news headlines | `/api/news/top?categories=business` |
| `/api/news/all` | Get all news with advanced search | `/api/news/all?search=technology` |
| `/api/news/similar/:uuid` | Get similar articles to a specific one | `/api/news/similar/cc11e3ab-ced0-4a42-9146-e426505e2e67` |
| `/api/news/uuid/:uuid` | Get a specific article by UUID | `/api/news/uuid/cc11e3ab-ced0-4a42-9146-e426505e2e67` |
| `/api/news/sources` | Get available news sources | `/api/news/sources?language=en` |

### Cache Management Endpoints

| Endpoint | Description | Example |
|----------|-------------|--------|
| `/api/cache/stats` | Get cache statistics | `/api/cache/stats` |
| `/api/cache/clear` | Clear all cache | `DELETE /api/cache/clear` |
| `/api/cache/clear/:type` | Clear cache by type | `DELETE /api/cache/clear/top` |

### Utility Endpoints

| Endpoint | Description |
|----------|-------------|
| `/health` | Health check endpoint |
| `/docs` | Interactive API documentation |
| `/docs.json` | OpenAPI specification |
| `/examples` | Usage examples |

## Testing

This project implements a comprehensive testing strategy with multiple layers of tests to ensure reliability and correctness.

### Test Types

1. **Unit Tests**
   - Isolated tests for individual components
   - Mocked dependencies for true unit isolation
   - Focused on business logic validation

2. **Controller Tests**
   - Verify controller behavior with mocked services
   - Test proper request handling and response formatting
   - Confirm error handling patterns are consistent

3. **Integration Tests**
   - Test the complete request-response cycle
   - Verify API endpoints with supertest
   - Mock external API calls for reliability

### Testing Best Practices

This project follows these testing best practices:

- **Isolation** - Tests don't depend on each other or external services
- **Repeatable** - Tests produce the same results on any environment
- **Fast** - Mocking external dependencies keeps tests efficient
- **Comprehensive** - Core functionality has high test coverage
- **Maintainable** - Tests follow consistent patterns and structure

### Running Tests

```bash
# Run all tests
npm test

# Run tests with coverage
npm run test:coverage

# Run a specific test file
npx jest path/to/test.ts
```

## Installation

```bash
# Clone the repository
git clone <repository-url>

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env file with your own API key and database URL

# Generate Prisma client
npx prisma generate

# Build the project
npm run build

# Start the server
npm start
```

## Configuration

Create a `.env` file with the following variables:

```
# News Aggregator API Environment Variables

# API Configuration
NEWS_API_TOKEN=your_api_token_here

# Server Configuration
PORT=3000
NODE_ENV=development

# Database Configuration
DATABASE_URL="file:./dev.db"
```

### Database Configuration

The application uses Prisma ORM with SQLite by default. The `DATABASE_URL` environment variable should point to your database file. For production, you can configure it to use PostgreSQL or MySQL.

## Development

```bash
# Run in development mode with hot reload
npm run dev

# Run linting
npm run lint

# Run tests
npm test
```

## Testing Infrastructure

The project includes a comprehensive test suite built with Jest and Supertest, covering API endpoints, database integration, and server connectivity checks.

### Running Tests

```bash
# Run all tests
npm test

# Run only API endpoint tests
npm run test:api

# Run only database integration tests
npm run test:db

# Run tests with coverage report
npm run test:coverage

# Run tests in watch mode (for development)
npm run test:watch

# Check server connectivity
npm run check-server
```

### Test Structure

- **API Endpoint Tests**: Located in `src/__tests__/api/`, these tests verify that all API endpoints return the expected responses, handle errors correctly, and apply appropriate filtering.

- **Database Integration Tests**: Located in `src/__tests__/database.test.ts`, these tests verify database connectivity, query execution, and transaction support.

- **Server Connectivity Script**: Located in `scripts/check-server.ts`, this script tests connectivity to a running server instance, verifying that critical endpoints are functioning correctly.

### Test Environment

Tests use a separate environment configuration specified by setting `NODE_ENV=test`. This ensures tests don't interfere with development or production environments. The setup includes:

- Automatic database connection setup and teardown
- Cache clearing between tests to ensure isolation
- Server startup and shutdown for endpoint testing

## Future Enhancements Todo List

### Priority 1: Foundation Improvements
- [x] **Caching System**
  - Implement in-memory caching
  - Cache frequently accessed data like top news and sources
  - Add cache invalidation strategies
  - Configure TTL (Time-To-Live) for different content types
  - Add cache management endpoints

- [x] **Data Persistence Layer**
  - Add database integration (SQLite with Prisma)
  - Create schema for storing enhanced article metadata
  - Implement repository pattern for data access
  - Add migration system for schema changes

- [x] **API Documentation**
  - Generate OpenAPI/Swagger specification
  - Add interactive API documentation
  - Create usage examples for each endpoint

## API Documentation

The API now includes comprehensive documentation using OpenAPI/Swagger:

- **Interactive API Documentation**: Available at `/docs` when the server is running
- **OpenAPI Specification**: JSON format available at `/docs.json`
- **Usage Examples**: Available at `/examples` with code samples in multiple languages

### How to Access the Documentation

1. Start the server with `npm run dev` or `npm start`
2. Open your browser to `http://localhost:3000/docs` for the interactive Swagger UI
3. Browse through the available endpoints, request parameters, and response formats
4. Reference implementation examples at `http://localhost:3000/examples`

### API Documentation Features

- Complete endpoint descriptions with parameters and response types
- Interactive "Try it out" capability to test endpoints directly
- Request/response schema definitions
- Code snippets for API consumption
- Examples of integrating with the MCP server architecture

### Priority 2: Security & Performance
- [ ] **Authentication & User Management**
  - Implement JWT-based authentication
  - Create user registration and login endpoints
  - Add role-based access control
  - Implement secure password handling

- [ ] **Rate Limiting**
  - Add request throttling middleware
  - Implement tiered access levels
  - Create fair usage policies
  - Add rate limit headers in responses

- [ ] **Monitoring & Analytics**
  - Set up request logging and monitoring
  - Add performance metrics collection
  - Create usage dashboards
  - Implement error tracking and alerting

### Priority 3: Enhanced Functionality
- [ ] **Content Processing**
  - Add text summarization capabilities
  - Implement entity extraction
  - Add sentiment analysis
  - Create topic categorization beyond source categories
  - Implement duplicate detection and grouping

- [ ] **Personalization**
  - Add user preference tracking
  - Create personalized endpoints based on reading history
  - Implement topic following functionality
  - Add recommendation engine

- [ ] **Webhooks & Real-time Updates**
  - Implement webhook registration
  - Create event-based notification system
  - Add SSE (Server-Sent Events) for real-time updates
  - Implement topic subscription functionality

## Contributing

Contributions to the News Aggregator API project are welcome! This project is designed to be a friendly, well-documented codebase for those interested in MCP server architecture, especially with AI consumption in mind.

### How to Contribute

1. Fork the repository
2. Create a new feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Run tests to ensure everything works (`npm test`)
5. Commit your changes (`git commit -m 'Add some amazing feature'`)
6. Push to the branch (`git push origin feature/amazing-feature`)
7. Open a Pull Request

### Code Style

Please follow the existing code style and patterns in the project. This project uses ESLint for code quality and formatting.

### Security Considerations

When contributing, please ensure:
- No API keys or credentials are hardcoded
- Environment variables are used for configuration
- No sensitive data is logged
- Input validation is properly implemented

### Testing

All new features should include appropriate tests. Please follow the existing testing patterns in the project.

- [ ] **Search History & Bookmarks**
  - Add saved searches functionality
  - Implement article bookmarking
  - Create reading history tracking
  - Add user collections feature

## Contributing

1. Fork the repository
2. Create your feature branch: `git checkout -b feature/my-new-feature`
3. Commit your changes: `git commit -am 'Add some feature'`
4. Push to the branch: `git push origin feature/my-new-feature`
5. Submit a pull request

## License

ISC