"A search for information about deep crawl web crawling and site auditing tools" matching MCP servers:

MCP server for web crawling, searching, and AI-powered content extraction, supporting single-page, batch, and full-site crawling along with text, news, image, book, and video search.

A local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.

Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.

Provides LLMs with real-time web search and content extraction capabilities, including text/news search, full-text URL reading, and targeted technical documentation search.

Unofficial MCP server wrapping crawl4ai that enables extraction and analysis of content from web pages, PDFs, Office documents, YouTube videos, and more, with AI-powered summarization and Google search integration to reduce token usage while preserving key information.

Provides web access capabilities for LLMs including search, fetching, content extraction, PDF reading, image viewing, and screenshots.

Provides DuckDuckGo web search and site fetching tools for Claude Desktop, enabling up to 10 search results and text content extraction from URLs.

Provides a web search tool with citations by running Claude Code CLI in an isolated Docker container, enabling search even in environments without direct Anthropic web search support.

MCP server for Cloudflare Browser Rendering Crawl API. Fetches and crawls web pages, returning clean Markdown optimized for LLM consumption.

A lean FastMCP server that wraps xAI's web_search and x_search tools for MCP hosts, enabling live web and X search with citations.

Enables MCP-compatible agents to perform web searches via the agent-web-search engine, returning ranked results with sources.

Enables site-wide crawling of internal links with BFS traversal, time/title/URL filtering, and page operations like summarization and link clicking, with results persisted to SQLite.

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

An open-source deep research MCP server that provides multi-source web search and synthesis with citations, enabling agents to perform citation-backed research using Qwen3-30B-A3B-Thinking and other models.

This MCP server enables free web search and page content extraction using real browsers, with support for multiple search engines and stealth browser backends. It provides tools for searching, fetching pages, and setting up the Camoufox stealth browser.

MCP server and CLI for querying Common Crawl via AWS Athena, enabling users to check a domain's presence in crawls, analyze dynamics, compare domains, and inspect page text.

Provides a local MCP server for AI-powered web crawling on Windows, enabling agents to directly crawl and extract web content without requiring WSL.

Enables intelligent code analysis and search across repositories using the CodeRank algorithm (inspired by PageRank) to identify critical modules, trace dependencies, find code hotspots, and perform context-aware keyword searches with importance-ranked results.

Enables AI assistants to perform real-time web and academic searches using Perplexity's Sonar API.

MCP server exposing web_search, web_fetch, and cluesift_domain_fetch tools with configurable Ollama API keys and HTTP or stdio transport.