"Information about web scraping or crawling" matching MCP servers:

An MCP server for safe web crawling and scraping of public and authenticated sites using Playwright and Crawlee. It provides tools for scraping single pages, crawling entire sites, validating sessions, and listing supported sites, with built-in security protections and adapter-based extensibility.

MCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.

Provides an LLM client with self-hosted web scraping capabilities including scraping, crawling, mapping, searching, structured data extraction, browser interaction, and deep research, without reliance on paid proxy or CAPTCHA services.

Thunderbit MCP lets AI agents search, scrape, and extract structured data from any website.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

Provides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.

MCP server for multi-engine web search and web page fetching, supporting parallel search, content extraction, and optional LLM-powered search summarization and deep search.

A headless web scraping server that extracts main content from web pages into Markdown, text, or HTML for AI and automation integration. It features per-domain rate limiting and robust error handling using Playwright and BeautifulSoup.

Enables an MCP client to read Discord servers, channels, threads, and messages, and to send messages, replies, reactions, and GIFs through an already-open Discord Web tab via a local Chrome extension and 127.0.0.1 bridge, without using a bot or the Discord HTTP API.

Enables compatible MCP clients to read JavaScript-rendered web pages as bounded Markdown or structured records, with URL policy and resource limits for safe, inspectable access.

Enables AI assistants to automate web interactions such as navigating pages, clicking elements, filling forms, querying the DOM, evaluating JavaScript, and inspecting network/console logs — all from a Termux terminal without Chromium, root, or proot.

A self-hosted MCP server that reads web pages and converts them to Markdown, with built-in web search and anti-crawl fallback using Playwright.

Exposes web search, site mapping, scraping, crawling, schema-validated structured extraction, and stateful Playwright browser interaction as MCP tools, routing every call through the same authenticated REST gateway used by the platform's HTTP clients.

Enables fetching, crawling, and browsing anti-bot protected web pages with real browser rendering, returning clean JSON or markdown, and solving captchas.

An MCP server enabling AI agents to access the internet via web scraping, crawling, searching, and structured data extraction using Firecrawl.

MCP server that parses web pages into clean markdown via Crawl4AI's HTTP API, exposing parse_page and health-check tools with configurable proxy and timeout.

A web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.

Enables Claude to fetch web pages from the user's own local network connection instead of AI datacenter IPs, including optional authenticated sessions, with verdicts for bot challenges, login walls, and other failures.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

An MCP server that runs Apify scraping actors with quality-gated validation, scoring, and retries, returning embedding-ready chunks for AI agents.