"Web scraping and crawling tools" matching MCP servers:

Enables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.

A Model Context Protocol server that enables AI assistants to perform advanced web scraping, crawling, searching, and data extraction through the Firecrawl API.

An MCP Server for Web scraping and Crawling, built using Crawl4AI

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

High-performance server enabling AI assistants to access web scraping, crawling, and deep research capabilities through Model Context Protocol.

Provides web crawling and RAG capabilities for AI agents, enabling scraping of websites, storing content in a vector database (Supabase), and performing semantic search over crawled data.

A Model Context Protocol server that allows LLMs to interact with web content through standardized tools, currently supporting web scraping functionality.

Provides AI agents and RAG pipelines with production-grade web search, deep research, news, academic, extraction, crawling, and source verification tools. Enables citation-anchored, deduplicated, fresh, and security-hardened evidence gathering from across the web.

Provides MCP tools for web scraping, including smart fetching, proxy rotation, and browser rendering to bypass anti-bot challenges and render JavaScript-heavy sites.

A fast, dependency-light MCP server that converts web pages into clean Markdown, structured metadata, and classified links for language models.

Enables agents to perform full-text search and read Hermes Agent documentation offline from a local cache, without scraping the site.

A production-ready Model Context Protocol server that enables language models to leverage AI-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing AI-powered web searches.

An MCP server that runs Apify scraping actors with quality-gated validation, scoring, and retries, returning embedding-ready chunks for AI agents.

An MCP proxy that embeds tool definitions and returns only the top-k semantically relevant tools per query, reducing context usage and routing calls to upstream MCP servers.

Enables AI agents to scrape, crawl, clean, chunk, and index website content into Azure AI Search for full-text retrieval.

An MCP server for SearchApi that provides Google web, news, scholar, and YouTube search as agent-ready MCP tools.

Web scraping, crawling, and structured data extraction for AI agents. 5 tools: scrape (clean markdown from any URL), crawl (entire sites), map (discover URLs), extract (structured JSON), and search. 833ms avg latency, single binary, self-hostable.

Enables AI assistants to search a curated database of 200+ AI tool cards with pricing, platforms, use cases, and source links, so users can ask which tool to use for a task and get grounded recommendations.

Turn YouTube, TikTok, X videos and websites into structured data. Skip the hassle of video transcription and data scraping. Our APIs help you build better software and AI products faster.

Enables AI assistants to perform comprehensive research by searching Google, mining Reddit discussions, scraping web content with JS rendering, and synthesizing findings with citations into structured context.