Conduct deep web research on any query using crawling, search, and AI analysis to gather comprehensive information from multiple sources.
MIT
751,542 tools. Updated 2026-11-06 15:20
"A search for information about deep crawl web crawling and site auditing tools" matching MCP tools:
- Search and fetch public web pages, or run Firecrawl search, scrape, and crawl to retrieve ranked results, readable content, or crawl job pages.Apache 2.0Destructive
- Perform AI-powered web searches to extract structured data from search results for research, competitive analysis, and multi-source information gathering.MIT
- Start multi-page web crawling to extract structured data with AI or convert content to markdown from a starting URL.MIT
- Crawl internal links across a site or analyze links on one page to find broken links, then return a reportId for retrieving the results.MIT
- Search Bing web results for a query, returning organic links, related queries, people-also-ask, news, video modules, and paginated pages. Use for general web search; site: queries are unsupported.MIT
Matching MCP Servers
- FlicenseAqualityFmaintenanceA deep web search MCP server using LinkUp API that provides a deep_search tool for performing deep web searches with optional max results.13-
- AlicenseNot gradedqualityCmaintenanceMCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.MIT
Matching MCP Connectors
One question searched in science, world news, Internet Archive, GitHub and discussions. Free.
Search the web and the news from an agent, with page text in the results when you ask for it.
- Retrieve results from an asynchronous web crawling operation by polling with a request ID until completion, providing structured data or markdown content from processed webpages.MIT
- Extract new data from every page saved by a finished crawl without revisiting the site, saving crawl costs while charging only AI credits. Returns a new jobId to track with spidra_get_job.MIT
- Trigger a browser-agent crawl to explore a web app's pages and UI states, then populate the backend knowledge graph for contextual testing and evaluation.Apache 2.0
- Crawl a website and audit each page for WCAG 2.2 accessibility violations using Axe, Pa11y, Lighthouse, or IBM engines. Returns per-page results and a GDS compliance summary.MIT
- Enumerate a website's URLs via sitemap, HTTP crawl, or hybrid—no rendering needed. Use to scope a site or audit sitemap exposure before extraction.MIT
- Search the web with natural language queries and date/site filters to retrieve current information and recent data, returning relevant results with snippets.MIT
- Find current information beyond training cutoffs with ranked web results, freshness filters, and country-specific search. Verify facts, check documentation, or research topics using live results.MIT
- Search videos, archived web pages, and documents in one call by title, channel, domain, author, subject, or tags. Use deep mode to search captions and page text.GPL 3.0
- Search the live web and extract Markdown content from top-ranked result pages in one step, giving AI agents current information and source snippets for queries without a target URL.MIT
- Search Apify and Crawlee docs via full-text queries to find relevant pages on platform features, SDKs, CLI, REST API, and web scraping libraries.MIT
- Retrieve current news and announcements from the web. Provides real-time market insights and breaking news for timely decision-making.MIT
- Ranks web-access providers by measured benchmarks (quality, latency, cost, error rate) for a given job like search, scrape, or crawl. Returns executable routes and honest caveats to help decide which API to use.MIT
- Crawl a website using breadth-first link traversal and return each page as clean Markdown. Use to ingest documentation sites or sections of a site.MIT
- Crawl any website recursively to collect per-page SEO data including titles, meta descriptions, headings, and issues for site-wide audits and content analysis.MIT
- Retrieve detailed information about any TigerGraph tool, including capabilities, usage examples, prerequisites, and related tools. Use it to understand how to apply a tool effectively.Apache 2.0