"Information about web scraping or crawling" matching MCP servers:

Provides an LLM client with self-hosted web scraping capabilities including scraping, crawling, mapping, searching, structured data extraction, browser interaction, and deep research, without reliance on paid proxy or CAPTCHA services.

Enables fetching, crawling, and browsing anti-bot protected web pages with real browser rendering, returning clean JSON or markdown, and solving captchas.

Enables live-web research by providing stateless browsing, link extraction, and page search capabilities via Playwright, allowing agents to fetch and analyze web content without session management.

MCP server that lets AI editors execute JavaScript, read console output, and simulate clicks/input on static web pages. Supports multiple browser pages and can be deployed remotely via HTTP transport.

Enables live website health checks including TLS, HTTPS, and security headers, returning an A-F grade with specific fixes.

MCP server for web scraping with natural language, enabling site crawling, page fetching, and extraction of emails, phones, links, and social profiles. Returns clean markdown, handles JavaScript rendering, and respects rate limits.

Enables intelligent web searching using SearXNG with content crawling via Creeper, then summarizes webpage content using LLM to avoid token limit issues. Supports smart filtering with domain blacklist/whitelist and optional LLM-based relevance filtering.

A powerful tool for fetching and extracting text content from web pages and APIs, supporting web scraping, REST API requests, and Google Custom Search integration.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

MCP server for multi-engine web search and web page fetching, supporting parallel search, content extraction, and optional LLM-powered search summarization and deep search.

Enables web search via DuckDuckGo and web content fetching through the Model Context Protocol, allowing AI assistants to search and retrieve web page content.

Enables AI assistants to automate web interactions such as navigating pages, clicking elements, filling forms, querying the DOM, evaluating JavaScript, and inspecting network/console logs — all from a Termux terminal without Chromium, root, or proot.

A high-performance search service that converts results from Google, Bing, and DuckDuckGo into structured JSON or Markdown. It features multi-layer depth crawling and uses the Camoufox anti-detection browser for reliable content extraction and fallback search logic.

Provides web search and page fetching capabilities via Serper.dev or DuckDuckGo, deployable on Render with SSE transport.

MCP server that fetches web pages and uses local Qwen models to answer questions about their content, with site-scoped search and map-reduce chunking for long pages.

Enables AI agents to rapidly drive and inspect real web pages through persistent browser sessions, using accessibility-tree snapshots and DevTools-grade diagnostics to identify and diagnose issues.

MCP server for querying Google Gemini Web from Claude Code and other MCP clients, using cookie-based authentication. Enables creating, continuing, listing, viewing, and deleting Gemini chat conversations with model and thinking mode control.

Enables AI agents to extract clean, structured web content (articles, tables, links, visual layouts) optimized for LLM token efficiency, with fast response times and optional JavaScript support.

Provides real-time system metrics and information through a Model Context Protocol interface, enabling access to CPU usage, memory statistics, disk information, network status, and running processes.

Enables AI agents to fetch, crawl, extract, and search web content through a unified set of tools, with automatic routing to specialized backends.