TimeLooker MCP Server

# TimeLooker MCP Server with x402 Payment Integration A Model Context Protocol (MCP) server that provides automated search monitoring functionality with x402 payment integration. Monitor search queries, detect new content using AI-powered duplicate detection, and automatically process payments for task creation. ## <� Architecture Overview The system consists of multiple components working together: ### Core Components - **`src/api/task_manager_api.py`** - FastAPI service with x402 payment middleware - **`src/api/task_manager_client.py`** - HTTP client with x402 payment support - **`src/mcp/searcher_mcp.py`** - MCP server exposing tools to Claude Desktop - **`src/core/search_engine.py`** - Web search execution and AI comparison - **`src/core/task_manager.py`** - Database operations and task management - **`src/core/models.py`** - SQLAlchemy database models ### Payment Integration - **x402 Protocol** - Automatic payments for task creation ($1.00 per task) - **Payment Address** - `0x671cE47E4F38051ba3A990Ba306E2885C2Fe4102` - **Network** - base-sepolia - **Free Endpoints** - All operations except task creation ## =' Setup Instructions ### Option A: Local Development Setup #### 1. Install Dependencies ```bash cd /path/to/MCP_Server uv sync ``` #### 2. Environment Configuration Copy the example environment file and configure it: ```bash cp .env.example .env ``` Edit `.env` with your configuration: ```env # x402 Payment Configuration PAY_TO_ADDRESS=0x671cE47E4F38051ba3A990Ba306E2885C2Fe4102 PRIVATE_KEY=your_ethereum_private_key_here # API Configuration TASK_MANAGER_API_URL=http://localhost:8000 # OpenAI Configuration OPENAI_API_KEY=your_openai_api_key_here # Database Configuration (optional) DATABASE_URL=sqlite:///timelooker.db ``` ### Option B: AWS Cloud Deployment #### 1. Prerequisites - AWS CLI configured with appropriate permissions - AWS CDK installed: `npm install -g aws-cdk` - Python dependencies: `uv sync` #### 2. Deploy Infrastructure ```bash # Deploy AWS infrastructure (RDS, Lambda roles, SES, etc.) python scripts/deploy_infrastructure.py ``` This creates: - PostgreSQL RDS database (db.t3.micro) - S3 bucket for email templates - AWS Secrets Manager for API keys - IAM roles for Lambda execution - SES email identity #### 3. Configure Secrets After deployment, update the secrets in AWS Secrets Manager: ```bash # Update OpenAI API key aws secretsmanager update-secret \ --secret-id "timelooker/openai/api-key" \ --secret-string '{"api_key":"your_openai_key_here"}' # Update X402 private key aws secretsmanager update-secret \ --secret-id "timelooker/x402/private-key" \ --secret-string '{"private_key":"your_private_key_here"}' ``` #### 4. Verify SES Email Go to AWS Console > SES > Verified identities and verify your sender email address. #### 5. Environment Configuration The deployment script creates `.env.aws` with the infrastructure details. Update it with your values: ```bash cp .env.aws .env # Edit .env to add your private key and sender email ``` #### 6. Test Secrets Integration Verify that the system can retrieve secrets from AWS: ```bash # Test secrets retrieval python scripts/test_secrets.py ``` This will show whether the system can automatically retrieve database credentials, API keys, and other secrets from AWS Secrets Manager. ### 3. Database Management #### Initialize Database ```bash # Basic initialization python scripts/init_db.py # Initialize with sample data python scripts/init_db.py --sample ``` #### Database Operations ```bash # Check schema version python scripts/init_db.py --version # Validate database integrity python scripts/init_db.py --validate # Run database migrations python scripts/init_db.py --migrate # Reset database (drop and recreate) python scripts/init_db.py --reset # Reset and create sample data python scripts/init_db.py --reset --sample ``` ### 4. Start the API Server ```bash python run_api_server.py # or uv run run_api_server.py ``` ### 5. Configure Claude Desktop Edit your Claude Desktop configuration file: - **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` - **Windows**: `%APPDATA%\Claude\claude_desktop_config.json` Choose the appropriate configuration based on your deployment mode: #### For Local Development: ```bash cp claude_desktop_config_local.json ~/Library/Application\ Support/Claude/claude_desktop_config.json ``` #### For AWS Cloud Deployment: ```bash cp claude_desktop_config_cloud.json ~/Library/Application\ Support/Claude/claude_desktop_config.json # Then edit the file to replace YOUR_ACCOUNT and endpoint values with actual values from deployment ``` #### Manual Configuration: You can also manually copy the contents from `claude_config_example.json` and add your values. **Important**: Replace `/absolute/path/to/MCP_Server` with your actual absolute path! ### 6. Restart Claude Desktop Completely restart Claude Desktop after editing the configuration. ## =� MCP Tools Available The MCP server provides 6 powerful tools for Claude Desktop: ### = **create_search_task** (Requires Payment) Create automated monitoring tasks for any search query - **Payment**: $1.00 per task creation - **Parameters**: query, frequency (min 1 min), email, runtime, sender email - **Cloud Mode**: Automatically deploys Lambda function + EventBridge schedule - **Local Mode**: Creates task in database for manual execution - **Example**: *"Create a search task to monitor AI Ethics job postings every hour for the next 3 days"* ### � **execute_search** (Free) Run a search for an existing task and get new results - **Parameters**: task_id - **Example**: *"Execute search for task 1"* ### =� **list_search_tasks** (Free) View all your active monitoring tasks - **Example**: *"Show me all my search tasks"* ### =� **get_task_status** (Free) Get detailed status and execution history for a task - **Parameters**: task_id, number of recent executions to show - **Example**: *"What's the status of task 2?"* ### =� **delete_search_task** (Free) Deactivate a monitoring task - **Parameters**: task_id - **Example**: *"Delete task 3"* ### =@ **search_preview** (Free) Preview search results without creating a task - **Parameters**: query, max_results - **Example**: *"Preview search results for 'Python developer remote jobs'"* ## =� Usage Examples Once configured, you can use natural language in Claude Desktop: ### Creating Tasks (Requires Payment) - *"Create a search task to monitor new iPhone releases every 2 hours for the next week"* - *"Set up monitoring for 'remote Python jobs' checking every 5 minutes for 1 hour"* - *"Monitor AI safety research papers, check daily for 2 weeks"* ### Managing Tasks (Free) - *"Show me all my active search tasks"* - *"What's the status of my first search task?"* - *"Execute a search for task 2"* - *"Delete the iPhone monitoring task"* ### Previewing (Free) - *"Preview search results for 'Machine Learning conferences 2025'"* - *"Show me what results I'd get for monitoring crypto news"* ## ( Key Features ### > **AI-Powered Duplicate Detection** - Uses OpenAI to identify genuinely new items vs. formatting variations - Significantly reduces false notifications - Considers content similarity, company, and location ### =� **x402 Payment Integration** - Automatic payment processing for task creation - Free access to all other operations - Ethereum-based payments on base-sepolia network ### = **Rich Search Results** - Finds items from multiple sources across the web - Extracts titles, descriptions, URLs, locations, and more - Structured data format for easy processing ### � **Flexible Monitoring** - Any search query (jobs, products, news, research, etc.) - Configurable frequency (1 minute to days) - Customizable runtime periods (1 minute to weeks) ### =� **Email Notifications** - Automatic notifications when new items are found - Structured email format with all item details - Configurable sender/recipient ### =� **Full History Tracking** - Complete execution history for each task - Performance metrics and error tracking - Easy status monitoring through Claude Desktop ### 🗄️ **Robust Database Management** - Automated schema migrations and version tracking - Database integrity validation and orphan detection - Consistent session management with automatic cleanup - Comprehensive CLI tools for database operations ## <� B Deployment Options ### Local Development Mode - **Setup**: Use `.env` with local database (SQLite) - **Task Execution**: Manual via MCP tools or scheduled scripts - **API Server**: Run locally with `python run_api_server.py` - **MCP Server**: Run locally with `python run_mcp_server.py` - **Database**: SQLite file - **Payments**: Still processed via x402 ### AWS Cloud Mode - **Setup**: Use `.env` with `DEPLOY_TO_CLOUD=true` - **Task Execution**: Automatic via Lambda functions + EventBridge - **Infrastructure**: RDS PostgreSQL, Lambda functions, SES, S3 - **Scaling**: Serverless, automatically handles multiple tasks - **Cost**: ~$15-30/month for typical usage - **Benefits**: - No manual task execution needed - Automatic cleanup after runtime expires - Professional email templates via SES - Scalable and fault-tolerant ### Hybrid Mode - **API Server**: Local development with cloud database - **Task Creation**: Creates Lambda functions for execution - **Best for**: Development while using production infrastructure ### Automation Options #### Option 1: Cron Job ```bash # Check every 5 minutes, run eligible tasks */5 * * * * cd /path/to/MCP_Server && python scripts/run_scheduled_tasks.py ``` #### Option 2: Background Service ```bash # Run continuously, checking every 60 seconds python scripts/run_scheduled_tasks.py --daemon --interval 60 ``` ## =' API Endpoints The FastAPI server provides these endpoints: - `POST /tasks/` - Create task (requires payment) - `GET /tasks/` - List active tasks - `GET /tasks/{task_id}` - Get task details - `DELETE /tasks/{task_id}` - Deactivate task - `POST /executions/` - Create execution record - `PUT /executions/{execution_id}` - Update execution - `GET /tasks/{task_id}/should-run` - Check if task should run - `POST /tasks/{task_id}/results` - Save search results - `GET /tasks/{task_id}/results` - Get previous results - `POST /tasks/{task_id}/notify` - Send email notification ## = Troubleshooting ### MCP Server Not Showing Up 1. Check Claude Desktop logs: `~/Library/Logs/Claude/mcp*.log` 2. Verify the absolute path in your configuration 3. Ensure all dependencies are installed 4. Restart Claude Desktop completely ### Payment Issues 1. Verify `PRIVATE_KEY` is set in `.env` 2. Ensure you have funds on base-sepolia network 3. Check that the private key is valid (without 0x prefix) ### Search Errors 1. Verify `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` is set and valid 2. Check internet connection 3. Look at API server logs for detailed error messages ### Database Issues 1. Check if `timelooker.db` exists and is writable 2. Run `python scripts/init_db.py --validate` to check database integrity 3. Run `python scripts/init_db.py --reset` to reinitialize if corrupted 4. Use `python scripts/init_db.py --version` to check schema version 5. Verify SQLAlchemy connection string in environment variables ## =� Task Frequency Guidelines - **Real-time Testing**: 1-5 minutes (short duration) - **Breaking News**: 5-30 minutes - **Job Postings**: 1-6 hours - **Product Releases**: 6-24 hours - **Research Papers**: 1-7 days **Note**: Very frequent checks are great for testing, but be mindful of OpenAI API costs for long-running tasks. ## = Related Files ### Project Organization ``` MCP_Server/ ├── src/ │ ├── api/ # HTTP API layer │ ├── core/ # Core business logic │ └── mcp/ # MCP server interface ├── tests/ # All test files ├── scripts/ # Utility scripts ├── run_mcp_server.py # MCP server entry point └── run_api_server.py # API server entry point ``` - **Testing**: `tests/test_search_quality.py`, `tests/quick_quality_test.py`, `tests/monitor_query_test.py` - **Lambda**: `scripts/lambda_function.py`, `tests/test_lambda.py` - **Automation**: `scripts/run_scheduled_tasks.py` - **Entry Points**: `run_mcp_server.py`, `run_api_server.py` ## 🧪 Testing The project includes a comprehensive test suite covering search quality, payment integration, and API functionality. ### Quick Test Run ```bash # Run all tests python tests/run_all_tests.py # Interactive test selection python tests/run_quality_tests.py ``` ### Test Categories #### 1. Search Quality Tests Validate search result quality and duplicate detection: ```bash python tests/quick_quality_test.py python tests/test_search_quality.py ``` #### 2. X402 Payment Integration Tests Test payment flows with mocked x402 client: ```bash python tests/test_x402_integration.py ``` **Tests include:** - TaskManager initialization with payment config - Create task with payment flow (mocked) - Free endpoints work without payment - Missing private key handling - Payment failure scenarios #### 3. API Integration Tests Test FastAPI endpoints and database integration: ```bash # Start API server first python run_api_server.py # In another terminal, run tests python tests/test_api_integration.py ``` **Tests include:** - Health endpoint functionality - Free endpoints (GET requests) - Payment-required endpoints (POST /tasks/) - Database model operations #### 4. Lambda Function Tests Test AWS Lambda compatibility: ```bash python tests/test_lambda.py ``` ### Test Configuration For API integration tests, ensure you have: - API server running on localhost:8000 - Valid environment variables in `.env` - Database initialized with `python scripts/init_db.py` ### Environment Variables for Testing ```env # Required for payment tests PRIVATE_KEY=your_test_private_key_here PAY_TO_ADDRESS=0x671cE47E4F38051ba3A990Ba306E2885C2Fe4102 X402_NETWORK=base-sepolia # Required for search tests OPENAI_API_KEY=your_openai_api_key_here ANTHROPIC_API_KEY=your_anthropic_api_key_here # Optional for testing DATABASE_URL=sqlite:///test.db LOG_LEVEL=INFO ``` ### Test Results The test suite validates: - ✅ Search result quality and consistency - ✅ AI-powered duplicate detection accuracy - ✅ Payment integration flows - ✅ API endpoint functionality - ✅ Database operations - ✅ Error handling and edge cases This system provides powerful search monitoring capabilities with seamless payment integration through the x402 protocol!