Discover available integrations and apps in Agentled to identify tools for building workflows, including LinkedIn enrichment, email finder, web scraping, and CRM connections.
MIT
215,089 tools. Last updated 2026-06-19 23:35
"Web scraping and crawling tools" matching MCP tools:
- Analyze a website's robots.txt file to determine crawl permissions and ensure compliance with ethical web scraping practices. Provides insights into allowed and disallowed paths for crawling.MIT
- Conduct deep web research on any query using crawling, search, and AI analysis to gather comprehensive information from multiple sources.MIT
- Conduct comprehensive web research by crawling, searching, and analyzing content to gather detailed information on any topic.MIT
- Generate critical instructions for web scraping tools to handle proxy rotation and bypass anti-bot measures when extracting data from websites.ISC
- Extract web content from URLs using Bittensor's decentralized scraping network. Pay per use via blockchain for automated data collection.
Matching MCP Servers
- AlicenseAqualityDmaintenanceProvides web access capabilities for LLMs including search, fetching, content extraction, PDF reading, image viewing, and screenshots.Last updated345MIT
- FlicenseAqualityDmaintenanceProvides local web search and content fetching capabilities for AI assistants, enabling them to search DuckDuckGo and extract clean text from web pages. All requests originate from the user's machine to ensure direct network control and bypass external proxies.Last updated2
Matching MCP Connectors
Web scraping, code review, content gen, sentiment. Zero Core Tools.
Web scraping for AI agents. Extract text and metadata from any URL worldwide. $0.005/page.
- Retrieve French tax information from cached data when web scraping fails, providing reliable access to official tax brackets and calculations for residents.
- Extract focused web content with optimized scraping, limited scrolls, and customizable image extraction for efficient data collection on Prysm MCP Server.
- Extract comprehensive web content, including images, using deep scraping techniques with customizable parameters such as scroll depth, image size, and pagination. Output data to a specified directory for thorough analysis.
- Create a new analysis suite and deploy an AI crawler to explore a web application from a starting URL, mapping its pages and structure. Optionally accepts login credentials, cookie preferences, and custom instructions.MIT
- Start multi-page web crawling to extract structured data with AI or convert content to markdown from a starting URL.MIT
- Execute multi-step web scraping workflows with AI automation, navigating websites, interacting with forms, and extracting structured data for complex scenarios requiring user simulation.MIT
- Control a web browser to click, type, scroll, take screenshots, and run JavaScript. Chain up to 20 actions per call for login flows, form submission, or scraping interactive pages.MIT
- Retrieve results from an asynchronous web crawling operation by polling with a request ID until completion, providing structured data or markdown content from processed webpages.MIT
- Search Apify and Crawlee documentation with full-text queries. Specify the source to find relevant pages and content snippets.MIT
- Discover pre-built scraping tools for any platform or use case by searching the Apify Store.MIT
- Discover all Novada tools by listing their names, descriptions, categories, and statuses. Filter by category to quickly find the right tool for your task.MIT
- Retrieve current news and announcements from the web. Provides real-time market insights and breaking news for timely decision-making.MIT
- Route requests through datacenter IPs for high-volume scraping of APIs and public data feeds without aggressive anti-bot protections. Supports geo-targeting and sticky IP sessions.MIT
- Extract and analyze hyperlinks from web pages, organizing URLs, anchor text, and contextual information into a structured format. Supports site mapping, SEO analysis, broken link checking, and targeted crawling preparation. Handles relative and absolute URLs with optional base URL and output limits.MIT