"A search for information about deep crawl web crawling and site auditing tools" matching MCP servers:

MCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.

MCP server for web crawling, searching, and AI-powered content extraction, supporting single-page, batch, and full-site crawling along with text, news, image, book, and video search.

Enables AI agents to extract structured content from URLs, search the web via DuckDuckGo, monitor site availability, and convert HTML to markdown without requiring API keys.

Provides DuckDuckGo web search and site fetching tools for Claude Desktop, enabling up to 10 search results and text content extraction from URLs.

Provides eight tools for web search, content extraction, screenshots, PDFs, JavaScript execution, crawling, and Wayback Machine snapshots and archiving.

Provides comprehensive search capabilities including web search, content extraction, news search, academic search, and AI-powered multi-source research. Enables natural language access to web content and research through a production-ready MCP server.

A local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

Zero-dependency MCP server that wraps OpenAI Codex's standalone search endpoint into codex_web_search and codex_web_research tools for Claude Code, enabling real-time web search and multi-step deep research independent of the underlying model.

Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.

Provides LLMs with real-time web search and content extraction capabilities, including text/news search, full-text URL reading, and targeted technical documentation search.

A self-hosted MCP server for web search with multi-engine support, combining search, URL fetching, and deep research along with proxy routing and deduplication.

Screenshot, visual-diff, and AI page-analysis API for AI agents. Capture any URL as PNG, JPEG, WebP, PDF, or HTML, diff two versions of a page to catch visual regressions, and get an AI summary of what a page contains.

Unofficial MCP server wrapping crawl4ai that enables extraction and analysis of content from web pages, PDFs, Office documents, YouTube videos, and more, with AI-powered summarization and Google search integration to reduce token usage while preserving key information.

An MCP server that enables Claude Code to perform web searches via Bing, read webpage content, and get current time, with a smart search skill for structured multi-source verification.

Provides web access capabilities for LLMs including search, fetching, content extraction, PDF reading, image viewing, and screenshots.

Multi-source web search MCP server with RRF fusion, 4-layer URL extraction, and provider health tracking.

Enables AI agents to capture full-page or viewport screenshots of any web page with options for ad removal, cookie banner blocking, and proxy country selection.

Enables web searching via DuckDuckGo and extracting readable content from any URL using Mozilla Readability, providing web context similar to Cursor's built-in functionality.

Enables free web searching using Google search results with no API keys required, returning structured results with titles, URLs, and descriptions.