"Information about web crawlers or related technology" matching MCP servers:

An MCP server that provides information about Utkarsh, including bio, skills, work experience, and portfolio projects, accessible via local stdio or remote HTTP with OAuth.

Provides live wait times and office details for all 60 Oregon DMV field offices by scraping the ODOT website, with no authentication required.

Enables extracting explicitly stated contract metadata, clauses, and obligations from user-supplied contract text without storing or modifying the data.

A self-hosted MCP server providing web search and URL fetching tools, running locally without external API keys or accounts.

Enables live website health checks including TLS, HTTPS, and security headers, returning an A-F grade with specific fixes.

Enables Claude and other LLMs to read and parse web content, with smart fallback strategies to bypass access restrictions (e.g., paywalls, Cloudflare) and output Markdown.

Read-only MCP server for WhatsApp Web allowing listing chats and reading recent messages, without sending or mutating WhatsApp state.

Provides web search and page fetch tools for LLM assistants, extracting content and enabling text search within fetched pages.

Enables web research through multi-provider search, documentation lookup, public code search, and clean Markdown extraction from static or JavaScript-rendered pages via five read-only tools.

Web search API for AI agents. Returns structured results with title, URL, and snippet; pay-per-call via x402 micropayments.

Enables web search via DuckDuckGo and web content fetching through the Model Context Protocol, allowing AI assistants to search and retrieve web page content.

Enables compatible MCP clients to read JavaScript-rendered web pages as bounded Markdown or structured records, with URL policy and resource limits for safe, inspectable access.

Enables users to select UI elements on localhost pages and send fix requests to MCP-compatible coding agents, which retrieve the captures and edit the code, with masking of sensitive values.

Enables AI agents to search DuckDuckGo and fetch web content with human-assisted CAPTCHA handling, designed for local use where humans can intervene when bot detection occurs.

Provides MCP tools to search and read social discussions from Reddit and X, and fetch pages blocked by bot protection, using a private Cloudflare Worker.

A self-hosted MCP server that reads web pages and converts them to Markdown, with built-in web search and anti-crawl fallback using Playwright.

Converts web pages to Markdown for MCP clients like Claude, with support for single and batch reads, caching, fallback rendering, and SSRF protection.

A Model Context Protocol server that provides real-time web search capabilities to AI assistants through pluggable search providers, currently integrated with the Brave Search API.

A fast, dependency-light MCP server that converts web pages into clean Markdown, structured metadata, and classified links for language models.

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.