Scrapy MCP Server

MIT License

Overview InspectNew Endpoints Schema Related Servers Reviews Score

scrape_multiple_webpages

Concurrently scrape multiple URLs using the Scrapy MCP Server. Extract data efficiently with configurable methods (auto, simple, Scrapy, Selenium) and handling for complex web structures.

Instructions

Scrape multiple webpages concurrently.

This tool allows you to scrape multiple URLs at once, which is much faster than scraping them one by one. All URLs will be processed concurrently.

Input Schema

Name	Required	Description	Default
`request`	Yes

Input Schema (JSON Schema)

{
  "$defs": {
    "MultipleScrapeRequest": {
      "description": "Request model for scraping multiple URLs.",
      "properties": {
        "extract_config": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for data extraction",
          "title": "Extract Config"
        },
        "method": {
          "default": "auto",
          "description": "Scraping method: auto, simple, scrapy, selenium",
          "title": "Method",
          "type": "string"
        },
        "urls": {
          "description": "List of URLs to scrape",
          "items": {
            "type": "string"
          },
          "title": "Urls",
          "type": "array"
        }
      },
      "required": [
        "urls"
      ],
      "title": "MultipleScrapeRequest",
      "type": "object"
    }
  },
  "properties": {
    "request": {
      "$ref": "#/$defs/MultipleScrapeRequest",
      "title": "Request"
    }
  },
  "required": [
    "request"
  ],
  "type": "object"
}

This server cannot be installed

Other Tools from Scrapy MCP Server

Related Tools

batch_crawl_urls
@flyanima/open-search-mcp
batch_webpage_scrape
@Bosegluon2/spider-mcp
crawl_web_page
@ascentkorea/hubble_mcp
get_page_links
@random-robbie/mcp-web-browser
fetch
@sparfenyuk/mcp-proxy
get_page_content
@YuheiNakasaka/scrapbox-mcp

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ThreeFish-AI/scrapy-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server