666,884 tools. Updated 2026-10-16 20:59
"Web crawling techniques and tools" matching MCP tools:
- Retrieve techniques used by software of a threat group via the Group → Software → Techniques relationship.Apache 2.0
- Retrieve details on known software, malware, or tools, including associated techniques and groups.MIT
- Discover web application endpoints and parameters by crawling with Katana, revealing parameterized URLs for further vulnerability scanning with Dalfox and SQLMap.MIT
- Retrieve Web Application Firewall bypass techniques for specific vulnerability types such as XSS, SQLi, and SSRF.GPL 3.0
- Create optimized web search queries tailored to specific research tasks, using structured strategies and advanced techniques to enhance information retrieval efficiency for AI agents.MIT
- Conduct comprehensive web research by crawling, searching, and analyzing content to gather detailed information on any topic.MIT
Matching MCP Servers
- AlicenseAqualityDmaintenanceProvides web search and content extraction tools for AI assistants, with zero configuration and swappable backends.631 npm1MIT
- AlicenseAqualityDmaintenanceProvides web access capabilities for LLMs including search, fetching, content extraction, PDF reading, image viewing, and screenshots.346MIT
Matching MCP Connectors
26 tools for AI agents: read any web page as clean Markdown, get metadata, structure and tech stack, and check redirects, SSL, security headers, robots.txt, sitemaps, feeds, WHOIS/DNS, SEO, broken links, email authentication (SPF, DMARC, DKIM) and SSO (SAML, OIDC), plus cron, regex, color, Unicode, time zone utilities and Hyperliquid market data. Hosted on Apify; authenticate with an Apify API token (Bearer). Pay per successful call ($0.001-$0.003).
Pay-per-call tools for agents: web page to Markdown, tech stack, email domain check (x402 USDC)
- Conduct deep web research on any query using crawling, search, and AI analysis to gather comprehensive information from multiple sources.MIT
- Create an analysis suite and launch an AI crawler to map a web app's structure from a starting URL, combining suite creation and crawling in one step.MIT
- Start multi-page web crawling to extract structured data with AI or convert content to markdown from a starting URL.MIT
- Retrieve results from an asynchronous web crawling operation by polling with a request ID until completion, providing structured data or markdown content from processed webpages.MIT
- Extract new data from a completed crawl without re-crawling pages. Submit a new prompt and receive a jobId to track progress, paying only for AI tokens.MIT
- Retrieve detailed product specifications including available placements, printing techniques, sizes, colors, and design requirements for Printful catalog items.MIT
- Search Apify and Crawlee docs via full-text queries to find relevant pages on platform features, SDKs, CLI, REST API, and web scraping libraries.MIT
- Retrieve detailed information about embedded TouchDesigner techniques, including optional code snippets and setup/workflow guidance.MIT
- Test web application file upload security by attempting to upload PHP shells using Content-Type bypass techniques and alternative extensions to identify vulnerabilities.MIT
- Crawls multiple web sources and uses AI synthesis to generate an executive research briefing with citations for a topic or complex question.MIT
- Discover all URLs from a website using sitemaps, CommonCrawl, or in-site crawling. Get a complete list for scraping, analysis, or site mapping.MIT
- Analyze patterns in Bitcoin Stamps to identify common libraries, frameworks, and coding techniques used across the ecosystem.MIT
- Conduct deep web research on complex queries using intelligent crawling, search, and LLM analysis to generate comprehensive insights from multiple sources.MIT
- Conduct deep web research on complex queries using intelligent crawling, search, and LLM analysis to generate comprehensive insights from multiple sources.MIT
- Analyze complex research questions by crawling multiple web sources and generating comprehensive LLM-based analysis.MIT