"A server for crawling web pages and finding subpages" matching MCP servers:

MCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.

Provides a local MCP server for AI-powered web crawling on Windows, enabling agents to directly crawl and extract web content without requiring WSL.

MCP server enabling Claude Code to capture screenshots of Windows applications through WSL2, allowing AI to diagnose UI errors and verify layouts.

A self-hosted MCP server that reads web pages and converts them to Markdown, with built-in web search and anti-crawl fallback using Playwright.

An MCP server for safe web crawling and scraping of public and authenticated sites using Playwright and Crawlee. It provides tools for scraping single pages, crawling entire sites, validating sessions, and listing supported sites, with built-in security protections and adapter-based extensibility.

A fault-tolerant, stealth-enabled Model Context Protocol (MCP) server for web searching and content fetching. Built for AI Agents (Cursor, Claude Code, OpenCode), it uses a stealth browser engine to fetch pages, dynamically handles SPAs/React, and converts bloat into token-optimized Markdown.

MCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.

Enables a locally-run LLM to search the web, fetch pages as markdown, make arbitrary HTTP requests, and optionally render pages with headless Chromium.

Enables AI agents to search private GitHub Pages by authenticating in a dedicated browser and retrieving indexed sections from a local SQLite database, with support for multiple sites and languages.

Enables AI agents to control a Playwright-driven browser through natural-language instructions, including page interaction and live browser task execution.

A custom MCP server that provides AI agents with tools to fetch web pages, extract readable article text, extract structured data by CSS selector, and check robots.txt permissions.

Enables compatible MCP clients to read JavaScript-rendered web pages as bounded Markdown or structured records, with URL policy and resource limits for safe, inspectable access.

Converts web pages to Markdown for MCP clients like Claude, with support for single and batch reads, caching, fallback rendering, and SSRF protection.

Local MCP server for A-share stock trading via Tonghuashun, offering account/position queries, buy/sell/cancel orders with risk controls and forced user confirmation; currently simulated with a reserved interface for real broker channels.

A local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.

A web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.

Enables Claude to fetch web pages from the user's own local network connection instead of AI datacenter IPs, including optional authenticated sessions, with verdicts for bot challenges, login walls, and other failures.

Relays a result screenshot to Slack so you can review a coding agent's output without opening the app.

Enables Codex to send text tasks to ChatGPT web, await responses, continue conversations, choose reasoning effort, and manage projects through MCP tools on macOS.

Enables reading Slack channels, messages, and threads using the web interface via a persistent browser session, designed for guest users without API access. Does not send messages.