"Information about web crawlers or related technology" matching MCP servers:

Enables AI agents to query OpenRouter model information including prices, ELO rankings, context, and perform comparisons.

MCP server that provides OpenRouter model pricing data, enabling price lookups, trending/cheapest lists, and model searches without an API key.

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

Provides live wait times and office details for all 60 Oregon DMV field offices by scraping the ODOT website, with no authentication required.

HandaaS的MCP服务器，提供全域资讯搜索、企业新闻、行业资讯、主题跟踪和企业动态监控能力，帮助用户开展行业研究和舆情分析。

Enables extracting explicitly stated contract metadata, clauses, and obligations from user-supplied contract text without storing or modifying the data.

Provides current weather information for any location and generates AI-powered responses using Google Gemini through FastMCP.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.

MCP server for web page fetching (converting to Markdown/text with automatic fallback between Tavily and Firecrawl) and web search via Tavily.

Enables AI agents to search and fetch high-quality information from multiple sources, including general web APIs, Wikipedia, arXiv, Hacker News, Stack Exchange, and Crossref, with optional pro-mode deep research and clean markdown page extraction.

Discovers related blockchain addresses and domain names for web3 identities across different platforms including Ethereum, Farcaster, Lens, and ENS using next.id's relation server data.

Enables web searching via DuckDuckGo and extracting readable content from any URL using Mozilla Readability, providing web context similar to Cursor's built-in functionality.

Provides web connectivity tools for searching the web via DuckDuckGo or SerpAPI, fetching URL content, and extracting readable text from web pages.

Enables web research through multi-provider search, documentation lookup, public code search, and clean Markdown extraction from static or JavaScript-rendered pages via five read-only tools.

A headless web scraping server that extracts main content from web pages into Markdown, text, or HTML for AI and automation integration. It features per-domain rate limiting and robust error handling using Playwright and BeautifulSoup.

Enables bounded web research through DuckDuckGo search and retrieval of public HTTPS pages as readable text, Markdown, JSON, or XML, with optional excerpt matching and strict safety validation.

Converts URLs into clean, LLM-ready markdown, respecting robots.txt and never bypassing anti-bot measures or paywalls.

A comprehensive web scraping MCP server with 26 tools for fetching, parsing, extracting, and assisting with web content, returning Markdown-formatted results.

Enables extracting clean, structured markdown from any URL—stripping nav, ads, and scripts—for RAG pipelines and AI research agents, with pay-per-call micropayments via x402.