Discover available integrations and apps in Agentled to identify tools for building workflows, including LinkedIn enrichment, email finder, web scraping, and CRM connections.
MIT
466,890 tools. Updated 2026-08-20 08:49
"Web scraping and crawling tools" matching MCP tools:
- Search web pages and retrieve content with optional scraping. Get SERP results or extract full page content in formats like markdown and HTML.MIT
- Analyze a website's robots.txt file to determine crawl permissions and ensure compliance with ethical web scraping practices. Provides insights into allowed and disallowed paths for crawling.MIT
- Extract focused web content with optimized scraping, limited scrolls, and customizable image extraction for efficient data collection on Prysm MCP Server.MIT
- Conduct comprehensive web research by crawling, searching, and analyzing content to gather detailed information on any topic.MIT
- Conduct deep web research on any query using crawling, search, and AI analysis to gather comprehensive information from multiple sources.MIT
Matching MCP Servers
- AlicenseAqualityBmaintenanceMCP server for web scraping with natural language, enabling site crawling, page fetching, and extraction of emails, phones, links, and social profiles. Returns clean markdown, handles JavaScript rendering, and respects rate limits.8MIT
- AlicenseAqualityBmaintenanceProvides web search and content extraction tools for AI assistants, with zero configuration and swappable backends.6291MIT
Matching MCP Connectors
40+ web scraping tools from Firecrawl, Bright Data, Jina, Olostep, ScrapeGraph, Notte, and Riveter. Scrape, crawl, screenshot, and extract from any website. Starts at $0.01/call. Get your API key at app.xpay.sh or xpay.tools
Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.
- Generate critical instructions for web scraping tools to handle proxy rotation and bypass anti-bot measures when extracting data from websites.ISC
- Extract comprehensive web content, including images, using deep scraping techniques with customizable parameters such as scroll depth, image size, and pagination. Output data to a specified directory for thorough analysis.MIT
- Extract web content from URLs using Bittensor's decentralized scraping network. Pay per use via blockchain for automated data collection.
- Clear stored scraping history to free up memory and ensure privacy by removing past web scraping records.MIT
- Retrieve French tax information from cached data when web scraping fails, providing reliable access to official tax brackets and calculations for residents.
- Create a new analysis suite and deploy an AI crawler to explore a web application from a starting URL, mapping its pages and structure. Optionally accepts login credentials, cookie preferences, and custom instructions.MIT
- Search the Apify Store to discover existing scraping tools for any platform or use case. Find pre-built crawlers, scrapers, and AI agents before coding your own.MIT
- Start multi-page web crawling to extract structured data with AI or convert content to markdown from a starting URL.MIT
- Retrieve results from an asynchronous web crawling operation by polling with a request ID until completion, providing structured data or markdown content from processed webpages.MIT
- Retrieve live bid, ask, and last price for a trading symbol from a connected broker. Get current market quote without web scraping.MIT
- Extract new data from a completed crawl without re-crawling pages. Submit a new prompt and receive a jobId to track progress, paying only for AI tokens.MIT
- Get instructions for obtaining an API key and using a web-access gateway that connects to 24 search, scraping, browser, and voice providers at provider-metered prices.MIT
- Search Apify and Crawlee documentation for relevant pages using full-text queries.MIT
- Ranks web-access providers by measured benchmarks (quality, latency, cost, error rate) for a given job like search, scrape, or crawl. Returns executable routes and honest caveats to help decide which API to use.MIT