"Web scraping and crawling tools" matching MCP servers:

Provides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.

An MCP server enabling AI agents to access the internet via web scraping, crawling, searching, and structured data extraction using Firecrawl.

Enables AI assistants to fetch public web pages as readable text or HTML, with caching, live refresh, and parallel job handling.

An APAC-native web scraping API for AI agents that provides tools for scraping, crawling, searching, and extracting structured data from websites, directly usable from MCP-compatible clients like Claude Desktop, Cursor, and Windsurf.

Integrates Firecrawl for web scraping, crawling, search, and content extraction capabilities. Supports single/batch scraping, URL discovery, structured data extraction, deep research, and AI-powered web analysis with automatic retries and rate limiting.

Integrates Firecrawl web scraping capabilities, enabling web content extraction, crawling, site mapping, search, and structured data extraction with automatic rate limiting and retry handling.

An MCP server that enables LLMs to drive a Chromium browser for web scraping, collecting structured data, and exporting to CSV.

Enables fetching any public web page through managed proxies, with optional JavaScript rendering, returning markdown, text, HTML, or structured JSON.

MCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.

Provides 27 MCP-native tools for web scraping, crawling, deep research, and autonomous extraction, delivering clean Markdown and structured JSON from any website.

Enables Claude Desktop to perform advanced web scraping and crawling operations, extracting structured data, analyzing website architectures, and discovering content relationships through natural conversation.

A portable MCP server that scrapes SPA/JS-rendered webpages and extracts structured API endpoint data. It uses headless Chromium to render JavaScript and provides tools for scraping, endpoint extraction, screenshots, and database seeding.

Enables web scraping, crawling, and content extraction by integrating with the Firecrawl API. It supports deep research, batch scraping, and automatic rate limiting for both cloud and self-hosted environments.

Provides AI agents live web access through scraping, multi-engine search, site mapping, crawling, SEO audits, and 31 data collectors via natural language tool calls.

Enables LLMs to fetch and process web page contents by providing tools to retrieve raw HTML, extract clean text, and convert web pages to markdown format.

⚡ 87% Token-compressed clean web scraping, YouTube AI transcripts, and PDF extraction for LLMs. Instant Free Trial with zero setup (No wallet/crypto required). Supports gasless agent vaults and EIP-712 on-chain oracle verification on Polygon, Base & Arbitrum.

A Model Context Protocol server that provides real-time web search capabilities to AI assistants through pluggable search providers, currently integrated with the Brave Search API.

Description: An MCP server with 15 tools covering web search, scraping, extraction, crawling, and autonomous data gathering via the SearchClaw API. Tagline: "The complete web data pipeline for AI agents — Search, Extract, Crawl in One API."

Enables AI agents to extract clean, structured web content (articles, tables, links, visual layouts) optimized for LLM token efficiency, with fast response times and optional JavaScript support.

An MCP-native pipeline for collecting structured intelligence on higher education institutions using the WHED schema, enabling scraping, extraction, validation, and saving of profiles.