# Scraper Maintenance MCP

A comprehensive Model Context Protocol (MCP) server for automating web scraper maintenance through intelligent browser inspection, selector generation, and code updates.

## 📁 Project Structure

```
mcp/
├── src/                    # TypeScript source files
│   ├── server.ts          # Main MCP server implementation
│   ├── browser-manager.ts # Browser automation and management
│   ├── selector-generator.ts # Selector generation and scoring
│   └── types.ts           # Type definitions
├── dist/                   # Compiled JavaScript files
│   ├── server.js          # Main MCP server (executable)
│   ├── browser-manager.js # Browser automation
│   ├── selector-generator.js # Selector intelligence
│   └── types.js           # Type definitions
├── config/                 # Configuration files
│   ├── test-config.json   # Test configuration
│   ├── claude-desktop-config.json # Claude Desktop setup
│   └── *.json             # Various scraper configurations
├── examples/               # Usage examples and documentation
├── docs/                   # Documentation files
├── scripts/                # Build and utility scripts
├── package.json           # Project configuration
└── tsconfig.json          # TypeScript configuration
```

## 🚀 Quick Start

### 1. Install Dependencies

```bash
cd mcp
npm install
```

### 2. Build the Project

```bash
npm run build
```

### 3. Run the Server

```bash
npm start
```

## 🛠️ Available MCP Tools

### Configuration Management
- `load_scraper_config` - Load scraper configuration files
- `update_config` - Update configurations with new selector mappings

### Browser Operations  
- `initialize_browser` - Launch browser (headless/visible mode)
- `navigate_to_page` - Navigate to target URLs
- `take_screenshot` - Capture debugging screenshots
- `close_browser` - Cleanup browser resources

### Element Inspection
- `inspect_field_manually` - Interactive visual element selection
- `auto_detect_field` - AI-powered automatic element detection
- `validate_selectors` - Test selector reliability and performance
- `generate_selectors` - Create multiple selector variations with scoring
- `test_extraction` - Test data extraction using current selectors

### Maintenance & Code Generation
- `run_maintenance_check` - Comprehensive scraper health analysis
- `generate_extractor_code` - Multi-language code generation

## 📖 Usage

### For Claude Desktop

Add to your Claude Desktop configuration:

```json
{
  "mcpServers": {
    "scraper-maintenance": {
      "command": "node",
      "args": ["/path/to/mcp/dist/server.js"],
      "env": {
        "NODE_ENV": "production"
      }
    }
  }
}
```

### For Cursor

Add to your Cursor MCP configuration:

```json
{
  "mcpServers": {
    "scraper-maintenance": {
      "command": "node",
      "args": ["/path/to/mcp/dist/server.js"],
      "env": {
        "NODE_ENV": "production"
      }
    }
  }
}
```

## 🔧 Development

### Build

```bash
npm run build
```

### Development Mode

```bash
npm run dev
```

### Test

```bash
npm test
```

## 📚 Documentation

- [Setup Guide](docs/SETUP.md)
- [Implementation Guide](docs/CURSOR_IMPLEMENTATION.md)
- [Workflow Examples](examples/workflow-example.md)
- [Advanced Usage](examples/advanced-usage.md)