"Resources for Learning Web Scraping" matching MCP servers:

Web scraper and token compressor that converts HTML to clean markdown with 70-80% fewer tokens. Single-page compression and multi-page BFS crawling with auto-fallback fetch modes.

MCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.

Enables reading and sending messages in Google Messages for web through a paired Chrome tab using chromefile, with tools for status, listing conversations, reading message history, and sending texts.

A fast web reader for AI assistants, served over MCP. It reads web pages and PDFs and returns only the passages that answer the question, not whole pages. It runs locally and keeps nothing.

Thunderbit MCP lets AI agents search, scrape, and extract structured data from any website.

MCP server for multi-engine web search and web page fetching, supporting parallel search, content extraction, and optional LLM-powered search summarization and deep search.

Enables compatible MCP clients to read JavaScript-rendered web pages as bounded Markdown or structured records, with URL policy and resource limits for safe, inspectable access.

A comprehensive web scraping MCP server with 26 tools for fetching, parsing, extracting, and assisting with web content, returning Markdown-formatted results.

Enables AI agents to search DuckDuckGo and fetch web content with human-assisted CAPTCHA handling, designed for local use where humans can intervene when bot detection occurs.

A self-hosted MCP server that reads web pages and converts them to Markdown, with built-in web search and anti-crawl fallback using Playwright.

Converts web pages to Markdown for MCP clients like Claude, with support for single and batch reads, caching, fallback rendering, and SSRF protection.

Enables AI-powered web searches using DuckDuckGo with automatic summarization via Google's Gemini AI. Supports both summary and detailed analysis modes for search results.

MCP server that parses web pages into clean markdown via Crawl4AI's HTTP API, exposing parse_page and health-check tools with configurable proxy and timeout.

A web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.

An open-source MCP server providing AI agents with neural web search via Exa and tiered web fetch (Exa, local browser, Firecrawl) as a drop-in replacement for built-in web tools, preserving provenance and guarding against SSRF.

Enables AI agents to extract clean LLM-ready markdown and readable articles from web pages, parse metadata, tables, links, feeds, and contact info, audit paywalls, SEO, and XSS, and access these tools via Streamable HTTP or a stdio bridge.

Enables Claude to fetch web pages from the user's own local network connection instead of AI datacenter IPs, including optional authenticated sessions, with verdicts for bot challenges, login walls, and other failures.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

An MCP server that runs Apify scraping actors with quality-gated validation, scoring, and retries, returning embedding-ready chunks for AI agents.

Enables agents to search the live web and fetch URLs as clean Markdown, returning ranked search results and paginated page content through two lightweight tools.

Wraps Xiaomi MiMo's web_search API into an MCP tool for Claude Code, enabling real-time web searches with location awareness, automatic retry, and timeout control.

Enables lightweight web search discovery and on-demand extraction of cleaned page text from supplied URLs, returning snippets, headings, links, and saved full text for longer pages.

Enables Codex to send text tasks to ChatGPT web, await responses, continue conversations, choose reasoning effort, and manage projects through MCP tools on macOS.

Enables reading Slack channels, messages, and threads using the web interface via a persistent browser session, designed for guest users without API access. Does not send messages.

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

Provides LLMs with real-time web search and content extraction capabilities, including text/news search, full-text URL reading, and targeted technical documentation search.

Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.

The MCP for the Web Speed Agent SDK that enables post-auth agents.

Enables private web search and webpage content extraction using a local SearxNG instance, prioritizing user privacy and autonomy.

MCP server that persists web fetch and search results as timestamped JSONL, enabling long-term archival and cross-domain search via unified-history-mcp.