Skip to main content
Glama
Scraper Maintenance MCP

Scraper Maintenance MCP

by fuahyo

Scraper Maintenance MCP

A comprehensive Model Context Protocol (MCP) server for automating web scraper maintenance through intelligent browser inspection, selector generation, and code updates.

šŸ“ Project Structure

mcp/
ā”œā”€ā”€ src/                    # TypeScript source files
│   ā”œā”€ā”€ server.ts          # Main MCP server implementation
│   ā”œā”€ā”€ browser-manager.ts # Browser automation and management
│   ā”œā”€ā”€ selector-generator.ts # Selector generation and scoring
│   └── types.ts           # Type definitions
ā”œā”€ā”€ dist/                   # Compiled JavaScript files
│   ā”œā”€ā”€ server.js          # Main MCP server (executable)
│   ā”œā”€ā”€ browser-manager.js # Browser automation
│   ā”œā”€ā”€ selector-generator.js # Selector intelligence
│   └── types.js           # Type definitions
ā”œā”€ā”€ config/                 # Configuration files
│   ā”œā”€ā”€ test-config.json   # Test configuration
│   ā”œā”€ā”€ claude-desktop-config.json # Claude Desktop setup
│   └── *.json             # Various scraper configurations
ā”œā”€ā”€ examples/               # Usage examples and documentation
ā”œā”€ā”€ docs/                   # Documentation files
ā”œā”€ā”€ scripts/                # Build and utility scripts
ā”œā”€ā”€ package.json           # Project configuration
└── tsconfig.json          # TypeScript configuration

Related MCP server: Selenium MCP Server

šŸš€ Quick Start

1. Install Dependencies

cd mcp
npm install

2. Build the Project

npm run build

3. Run the Server

npm start

šŸ› ļø Available MCP Tools

Configuration Management

  • load_scraper_config - Load scraper configuration files

  • update_config - Update configurations with new selector mappings

Browser Operations

  • initialize_browser - Launch browser (headless/visible mode)

  • navigate_to_page - Navigate to target URLs

  • take_screenshot - Capture debugging screenshots

  • close_browser - Cleanup browser resources

Element Inspection

  • inspect_field_manually - Interactive visual element selection

  • auto_detect_field - AI-powered automatic element detection

  • validate_selectors - Test selector reliability and performance

  • generate_selectors - Create multiple selector variations with scoring

  • test_extraction - Test data extraction using current selectors

Maintenance & Code Generation

  • run_maintenance_check - Comprehensive scraper health analysis

  • generate_extractor_code - Multi-language code generation

šŸ“– Usage

For Claude Desktop

Add to your Claude Desktop configuration:

{
  "mcpServers": {
    "scraper-maintenance": {
      "command": "node",
      "args": ["/path/to/mcp/dist/server.js"],
      "env": {
        "NODE_ENV": "production"
      }
    }
  }
}

For Cursor

Add to your Cursor MCP configuration:

{
  "mcpServers": {
    "scraper-maintenance": {
      "command": "node",
      "args": ["/path/to/mcp/dist/server.js"],
      "env": {
        "NODE_ENV": "production"
      }
    }
  }
}

šŸ”§ Development

Build

npm run build

Development Mode

npm run dev

Test

npm test

šŸ“š Documentation

Available Tools

13 tools
auto_detect_fieldC

Automatically detect elements for a field using AI-like heuristics

Inputs: fieldName (required), strategies.

close_browserA

Close the browser and cleanup resources

Inputs: no named parameters.

generate_extractor_codeC

Generate extractor code based on current configuration

Inputs: language, outputPath.

generate_selectorsC

Generate multiple selector variations for an element

Inputs: elementData (required).

initialize_browserC

Initialize browser for inspection with custom options

Inputs: timeout, headless, viewport, userAgent.

inspect_field_manuallyB

Start manual inspection for a specific field using visual element selection

Inputs: fieldName (required).

load_scraper_configC

Load a scraper configuration file

Inputs: configPath (required).

run_maintenance_checkC

Run comprehensive maintenance check on scraper configuration

Inputs: url (required), configPath.

take_screenshotC

Take a screenshot of the current page

Inputs: outputPath.

test_extractionC

Test data extraction using current selectors

Inputs: url (required), selectors.

update_configA

Update scraper configuration with new selector mappings

Inputs: mappings (required), outputPath.

validate_selectorsB

Validate existing selectors and check their reliability

Inputs: selectors (required), url.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 13 tools

    Disambiguation5/5

    Each tool has a distinct purpose: browser lifecycle (initialize, navigate, close), config management (load, update), field inspection (manual vs. auto), selector operations (validate, generate), and testing/maintenance (test, audit, code generation). Even similar tools like inspect_field_manually and auto_detect_field are clearly differentiated by manual vs. automatic methodology.

    Naming Consistency5/5

    All tool names follow a consistent verb_noun pattern (e.g., load_scraper_config, navigate_to_page, generate_selectors). No mixed conventions or vague verbs; each name clearly indicates its action and target.

    Tool Count5/5

    With 13 tools, the server is well-scoped for scraper maintenance. Every tool supports the core workflow without redundancy, and the count falls comfortably within the ideal 3-15 range.

    Completeness5/5

    The tool set covers the full maintenance lifecycle: load configuration, inspect and detect fields, validate/generate selectors, test extraction, update config, run comprehensive checks, and generate final code. Minor features like exporting configs or handling pagination are absent, but the core workflow has no dead ends.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      D
      maintenance
      MCP server for crawling and extracting data from web pages using Selenium with CSS/XPath selectors. Supports 17 tools including smart extraction, pagination, infinite scroll, and screenshots.
      3
      -
    • A
      license
      Not graded
      quality
      D
      maintenance
      MCP server for web scraping and browser automation, enabling AI agents to extract clean, token-efficient content from web pages.
      1
      MIT
    • A
      license
      C
      quality
      C
      maintenance
      MCP server for browser automation that drives Chrome via an extension, preserving login state and offering 45 tools for navigation, interaction, scraping, and screenshots.
      53
      5
      MIT