"Information about web crawlers or related technology" matching MCP servers:

MCP server that provides OpenRouter model pricing data, enabling price lookups, trending/cheapest lists, and model searches without an API key.

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

HandaaS的MCP服务器，提供全域资讯搜索、企业新闻、行业资讯、主题跟踪和企业动态监控能力，帮助用户开展行业研究和舆情分析。

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.

MCP server for web page fetching (converting to Markdown/text with automatic fallback between Tavily and Firecrawl) and web search via Tavily.

Search and retrieve transcripts from 488 episodes of the Big Technology Podcast, enabling queries on tech topics and full episode access via Claude.

Enables AI agents to search and fetch high-quality information from multiple sources, including general web APIs, Wikipedia, arXiv, Hacker News, Stack Exchange, and Crossref, with optional pro-mode deep research and clean markdown page extraction.

Enables a locally-run LLM to search the web, fetch pages as markdown, make arbitrary HTTP requests, and optionally render pages with headless Chromium.

A self-hosted MCP server providing web search and URL fetching tools, running locally without external API keys or accounts.

Discovers related blockchain addresses and domain names for web3 identities across different platforms including Ethereum, Farcaster, Lens, and ENS using next.id's relation server data.

Persists web fetch and search results as timestamped JSONL files, indexed for full-text search, enabling users to retrieve and search across archived web pages even after they are removed or changed.

Provides a web search tool with citations by running Claude Code CLI in an isolated Docker container, enabling search even in environments without direct Anthropic web search support.

Comprehensive web research toolkit with 13 tools for searching (via SearXNG), crawling, package discovery, GitHub metrics, error translation, API documentation lookup, data extraction, technology comparison, and service status checking.

A local MCP server for semantic search over RSS/Atom feeds, providing tools for ranked search, trending topics, digests, and feed health monitoring without requiring an API key.

Provides local web search and content fetching capabilities for AI assistants, enabling them to search DuckDuckGo and extract clean text from web pages. All requests originate from the user's machine to ensure direct network control and bypass external proxies.

A local MCP server providing web search, page extraction, and safe browser automation tools for Hermes, Claude Code, and other MCP clients.

Enables web searching via DuckDuckGo and extracting readable content from any URL using Mozilla Readability, providing web context similar to Cursor's built-in functionality.

Provides web connectivity tools for searching the web via DuckDuckGo or SerpAPI, fetching URL content, and extracting readable text from web pages.

Enables web research through multi-provider search, documentation lookup, public code search, and clean Markdown extraction from static or JavaScript-rendered pages via five read-only tools.