"Web scraping and crawling tools" matching MCP servers:

MCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.

Provides web access capabilities for LLMs including search, fetching, content extraction, PDF reading, image viewing, and screenshots.

Provides an LLM client with self-hosted web scraping capabilities including scraping, crawling, mapping, searching, structured data extraction, browser interaction, and deep research, without reliance on paid proxy or CAPTCHA services.

Provides local web search and content fetching capabilities for AI assistants, enabling them to search DuckDuckGo and extract clean text from web pages. All requests originate from the user's machine to ensure direct network control and bypass external proxies.

An MCP Server for Web scraping and Crawling, built using Crawl4AI

Provides browser automation and web scraping as MCP tools, enabling autonomous URL ingestion, crawling, extraction, and anti-bot handling with interactive browser control.

Built as a Model Context Protocol (MCP) server that provides advanced web search, content extraction, web crawling, and scraping capabilities using the Firecrawl API.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

Efficient web scraping MCP server with a two-step workflow for token-efficient content extraction. Supports previewing HTML structure, targeted scraping, and entire documentation site crawling.

A lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.

A custom MCP server that provides AI agents with tools to fetch web pages, extract readable article text, extract structured data by CSS selector, and check robots.txt permissions.

A comprehensive web scraping MCP server with 26 tools for fetching, parsing, extracting, and assisting with web content, returning Markdown-formatted results.

A Model Context Protocol server that allows LLMs to interact with web content through standardized tools, currently supporting web scraping functionality.

An MCP server enabling AI agents to access the internet via web scraping, crawling, searching, and structured data extraction using Firecrawl.

An MCP server for safe web crawling and scraping of public and authenticated sites using Playwright and Crawlee. It provides tools for scraping single pages, crawling entire sites, validating sessions, and listing supported sites, with built-in security protections and adapter-based extensibility.

Provides AI agents and RAG pipelines with production-grade web search, deep research, news, academic, extraction, crawling, and source verification tools. Enables citation-anchored, deduplicated, fresh, and security-hardened evidence gathering from across the web.

Multi-engine web search and content extraction MCP server with 10 sources, 4-way parallel queries, and content extraction via Jina Reader, no API keys required.

A local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.

MCP server that parses web pages into clean markdown via Crawl4AI's HTTP API, exposing parse_page and health-check tools with configurable proxy and timeout.

A web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.