"How to scrape a website" matching MCP servers:

Enables AI agents to scrape, crawl, clean, chunk, and index website content into Azure AI Search for full-text retrieval.

Fetches and analyzes website content to provide business context to AI agents, designed for Voice AI in GoHighLevel.

A website audit MCP server that provides free tools to check on-page SEO elements like titles, descriptions, and images across up to 20 pages, and also validates robots.txt, sitemap.xml, and llms.txt accessibility.

Enables AI agents to fetch one or many web pages concurrently and receive clean, LLM-ready Markdown with navigation, ads and boilerplate stripped away, plus extracted metadata such as title, description, author, Open Graph tags and canonical URL. It runs locally over stdio with no API key or account, requiring only a single command.

Provides a tool to download entire websites using wget. It preserves the website structure and converts links to work locally.

Fetches website content and converts it to Markdown format with AI-powered content cleanup, ad removal, and full OpenAPI/Swagger specification support for easy processing by AI assistants.

Thunderbit MCP lets AI agents search, scrape, and extract structured data from any website.

Enables AI agents to control a Playwright-driven browser through natural-language instructions, including page interaction and live browser task execution.

Shows how AI sees your website

根据关键词获取小红书的热帖数据，需要钉钉扫码登录，配合 Chrome 使用。

Enables deterministic bilingual website intelligence audits by URL, returning fixed English or Spanish findings without ever contacting the submitted site.

A lightweight MCP server implementation that provides a simple tool for fetching website content, demonstrating how tools can be exposed and interacted with using the MCP protocol.

Enables converting webpages into structured JSON using CSS-selector schemas, with anti-bot handling and support for protected pages.

Enables AI-driven job application automation for LinkedIn and SEEK platforms with intelligent cover letter generation, automated application submission, and application tracking management. Supports anti-detection measures and complies with platform usage policies for safe job hunting automation.

Extracts clean text content and metadata from website URLs using Beautiful Soup through a standardized interface for AI agents. It supports both Bearer token authentication and a blockchain-based pay-per-use protocol for flexible access.

Fetches a URL and returns the main content as clean Markdown, using plain HTTP when possible and headless Chromium for JavaScript-rendered or bot-protected pages.

Enables AI assistants to perform web and image searches through a Google Programmable Search Engine, with pagination, language/country filters, and safe search.

Enables browser automation through Surfsky's cloud-based antidetect browser, allowing users to scrape pages, navigate and interact with websites, run JavaScript, take screenshots, and manage isolated sessions with custom profiles, proxies, and fingerprint settings.

MCP server for remote web scraping, providing a scrape_web tool that fetches pages and extracts content as markdown or JSON, supporting CSS selectors.

Enables AI agents to scrape Google Maps business leads, including contact details, emails, websites, and social profiles, and export them as CSV or XLSX.