"Scrapy" matching MCP servers:

A general-purpose MCP server for crawling and extracting structured data from any website. Supports tools for crawling, single-page extraction, search-and-crawl, and schema extraction.

A powerful web scraping MCP server built on Scrapy and FastMCP that supports multiple scraping methods (HTTP, Scrapy, browser automation), anti-detection techniques, form handling, and concurrent crawling. Designed for commercial environments with enterprise-grade features like intelligent retry mechanisms, performance monitoring, and configurable data extraction.

Enables AI assistants to access Twitter/X data, including user profiles, tweets, trends, and more, through the ScrapeBadger API.

Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC micropayments via the x402 protocol on Base

A Model Context Protocol server that enables AI clients to interact with virtual Ubuntu desktops, allowing them to browse the web, run code, and control instances through mouse/keyboard actions and bash commands.

This Serper MCP Server supports search and webpage scraping, and all the most recent parameters introduced by the Serper API, like location.

Enables web search via Serper API with advanced search operators and webpage scraping capabilities to extract content in plain text or markdown format.

A comprehensive web scraping server that transforms web content into clean, agent-ready Markdown with automatic citations and efficient caching. It features a robust suite of tools for metadata extraction, sentiment analysis, SEO auditing, and security scanning while strictly adhering to robots.txt policies.

Household-aware kitchen brain for AI agents: manage pantry inventory with freshness tracking, shopping lists, recipe collections with cook notes and per-diner ratings, dietary profiles with allergen safety, and kitchen equipment — all through 27 tools with OAuth 2.1 authentication. Includes a free tool for ingredient-based recipe generation without an account (accounts are free!).

SEO MCP connects AI assistants to Search Console, technical SEO audits, crawl insights, Core Web Vitals, and uptime data.

Enables AI agents to extract clean, structured content from any web page via a pay-per-call API or with a Forge key.

Service name: Scrape Agent Service base URL: https://scrapeagent.xyz Website URL: https://scrapeagent.xyz Email: your email Category: Data Description: Pay-per-use web scraping API built on x402. Extract clean text, links or HTML from any URL for $0.01 USDC on Base. No API key, no subscription needed. Perfect for AI agents that need web data on demand. Endpoint paths: /api/scrape/x402

MCP server that lets AI agents scrape web pages as clean markdown for a micro-payment in USDC, using the x402 protocol on Base network. No API keys or signup required.

MCP server for structured web data access, enabling local-market research and lead-list enrichment by returning business names, locations, ratings, and review signals from concrete queries.

An authless Cloudflare Workers server that provides tools to scrape, analyze, and answer questions about websites using Cloudflare's Browser Rendering and AI capabilities.

An MCP server for AI-agent web scraping that enforces robots.txt compliance and requires user confirmation of scrape plans, packaging scraped data with provenance evidence.