MCP server for fetching web pages without robots.txt restrictions, supporting all standard HTTP methods, custom headers, body, redirect control, and timeout settings.
A self-hosted MCP server offering unlimited web search, scraping, and content extraction tools with full-content reads, no truncation, and zero per-request cost.
A powerful web scraping MCP server built on Scrapy and FastMCP that supports multiple scraping methods (HTTP, Scrapy, browser automation), anti-detection techniques, form handling, and concurrent crawling. Designed for commercial environments with enterprise-grade features like intelligent retry mechanisms, performance monitoring, and configurable data extraction.
A general-purpose MCP server for crawling and extracting structured data from any website. Supports tools for crawling, single-page extraction, search-and-crawl, and schema extraction.