A
licenseA
qualityC
maintenanceEnables AI agents to scrape, crawl, clean, chunk, and index website content into Azure AI Search for full-text retrieval.
5
MIT
72,221 servers. Updated
Search Relevance ↓
Recent Usage ↓
Date Added ↓
Date Updated ↓
Weekly Downloads ↓
GitHub Stars ↓
Recent GitHub Stars ↓
Matching MCP tools:
Matching MCP Connectors:
net.agri-ja
dev.workers.o1labs.sterdan-siterdan-mcp
top.zheng-he
website.specification.mcp
"How to scrape a website" matching MCP servers:
- FlicenseAqualityDmaintenanceFetches and analyzes website content to provide business context to AI agents, designed for Voice AI in GoHighLevel.1
- AlicenseAqualityCmaintenanceA website audit MCP server that provides free tools to check on-page SEO elements like titles, descriptions, and images across up to 20 pages, and also validates robots.txt, sitemap.xml, and llms.txt accessibility.2MIT
- AlicenseAqualityDmaintenanceEnables scraping of emails, phone numbers, and social profile links from website domains. Supports batch processing of up to 20 domains and can find company websites by keyword/company name.2MIT
- FlicenseBqualityDmaintenanceProvides a tool to download entire websites using wget. It preserves the website structure and converts links to work locally.1156
- AlicenseBqualityDmaintenanceFetches website content and converts it to Markdown format with AI-powered content cleanup, ad removal, and full OpenAPI/Swagger specification support for easy processing by AI assistants.4124MIT
- Alicense-qualityBmaintenanceThunderbit MCP lets AI agents search, scrape, and extract structured data from any website.13MIT
- Alicense-qualityCmaintenanceMCP server that fetches and converts website pages into clean markdown for agents, with sitemap-based discovery, disk caching, and polite crawling (robots.txt, rate limiting).2MIT
- Alicense-qualityDmaintenanceClone, scrape and crawl any website. Free Firecrawl alternative with no API keys or rate limits.9MIT
- Alicense-qualityBmaintenanceMCP server that lets AI agents scrape web pages as clean markdown for a micro-payment in USDC, using the x402 protocol on Base network. No API keys or signup required.MIT
- Flicense-qualityDmaintenanceAn authless Cloudflare Workers server that provides tools to scrape, analyze, and answer questions about websites using Cloudflare's Browser Rendering and AI capabilities.6
- Flicense-qualityCmaintenanceProvides LLMs with real-time weather forecasts for Israeli cities using browser automation with Playwright to scrape live data from a weather website.
- Flicense-qualityFmaintenanceA lightweight MCP server implementation that provides a simple tool for fetching website content, demonstrating how tools can be exposed and interacted with using the MCP protocol.
- Flicense-qualityCmaintenance基于MCP架构的微信公众号文章爬虫系统，支持AI智能体通过Selenium自动抓取微信公众号文章内容和图片，实现文章内容的智能分析。41
- Flicense-qualityDmaintenanceEnables AI-driven job application automation for LinkedIn and SEEK platforms with intelligent cover letter generation, automated application submission, and application tracking management. Supports anti-detection measures and complies with platform usage policies for safe job hunting automation.
- Flicense-qualityDmaintenanceExtracts clean text content and metadata from website URLs using Beautiful Soup through a standardized interface for AI agents. It supports both Bearer token authentication and a blockchain-based pay-per-use protocol for flexible access.
- AlicenseAqualityBmaintenanceFetches a URL and returns the main content as clean Markdown, using plain HTTP when possible and headless Chromium for JavaScript-rendered or bot-protected pages.1MIT
- AlicenseAqualityAmaintenanceReal-time X (Twitter) data platform with 2 MCP tools covering 120+ REST API endpoints. Search tweets, look up users, get timelines, extract followers/likes/retweets in bulk, monitor accounts, run giveaway draws, and perform write actions (tweet, like, retweet, follow, DM). OAuth 2.1 authentication with PKCE.22694178MIT
