"A server for crawling web pages and finding subpages" matching MCP servers:

MCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.

A self-hosted MCP server that reads web pages and converts them to Markdown, with built-in web search and anti-crawl fallback using Playwright.

A fault-tolerant, stealth-enabled Model Context Protocol (MCP) server for web searching and content fetching. Built for AI Agents (Cursor, Claude Code, OpenCode), it uses a stealth browser engine to fetch pages, dynamically handles SPAs/React, and converts bloat into token-optimized Markdown.

Enables a locally-run LLM to search the web, fetch pages as markdown, make arbitrary HTTP requests, and optionally render pages with headless Chromium.

Enables AI agents to search private GitHub Pages by authenticating in a dedicated browser and retrieving indexed sections from a local SQLite database, with support for multiple sites and languages.

Enables web research through multi-provider search, documentation lookup, public code search, and clean Markdown extraction from static or JavaScript-rendered pages via five read-only tools.

Provides comprehensive A-share (Chinese stock market) data including stock information, historical prices, financial reports, macroeconomic indicators, technical analysis, and valuation metrics through the free Baostock data source.

This MCP server provides tools for AI agents to search the web, fetch page content, and query specific elements from pages using DuckDuckGo.

A local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.

Provides a unified web search tool for AI agents by running multiple search providers concurrently and returning normalized results. It integrates with Claude Code, Codex CLI, OpenCode, and Hermes.

Enables private web search and webpage content extraction using a local SearxNG instance, prioritizing user privacy and autonomy.

An MCP server for searching and downloading books from Library Genesis, supporting EPUB, MOBI, PDF, and more through natural language queries.

A self-hosted MCP server for web search with multi-engine support, combining search, URL fetching, and deep research along with proxy routing and deduplication.

Odysseus Web MCP is a standalone Model Context Protocol (MCP) server for safe public-web search and URL fetching.

MCP server for web page fetching (converting to Markdown/text with automatic fallback between Tavily and Firecrawl) and web search via Tavily.

Read-only MySQL MCP server that lets AI agents list tables, describe schemas, and run SELECT/SHOW/EXPLAIN queries with a row cap, bound to a single database for safety.

A local-first MCP server for web search and documentation fetching, providing free DuckDuckGo search, URL-to-markdown conversion, and library docs from npm, PyPI, crates.io, Go, and GitHub with caching.

A self-hosted MCP server providing web search and URL fetching tools, running locally without external API keys or accounts.