"Information about web crawlers or scraping techniques" matching MCP servers:

An MCP server that provides information about Utkarsh, including bio, skills, work experience, and portfolio projects, accessible via local stdio or remote HTTP with OAuth.

Enables AI agents to query OpenRouter model information including prices, ELO rankings, context, and perform comparisons.

Provides live wait times and office details for all 60 Oregon DMV field offices by scraping the ODOT website, with no authentication required.

MCP server that provides OpenRouter model pricing data, enabling price lookups, trending/cheapest lists, and model searches without an API key.

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

Enables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.

HandaaS的MCP服务器，提供全域资讯搜索、企业新闻、行业资讯、主题跟踪和企业动态监控能力，帮助用户开展行业研究和舆情分析。

Enables running the lead scoring and outbound pipeline by talking to Claude, including scraping healthcare job postings, scoring against buyer ICP, gating on company size, and managing sequences.

Enables extracting explicitly stated contract metadata, clauses, and obligations from user-supplied contract text without storing or modifying the data.

Enables MCP-compatible clients to scrape and extract data from live websites using a headless Chromium browser, with support for CSS selectors, link extraction, table extraction, and Amazon product pages.

Enables web page scraping via Jina reader API and searching FastMCP documentation using minsearch. Supports fetching markdown content from URLs and querying indexed documentation files.

Thunderbit MCP lets AI agents search, scrape, and extract structured data from any website.

Provides current weather information for any location and generates AI-powered responses using Google Gemini through FastMCP.

Enables Codex to send text tasks to ChatGPT web, await responses, continue conversations, choose reasoning effort, and manage projects through MCP tools on macOS.

MCP server that lets AI editors execute JavaScript, read console output, and simulate clicks/input on static web pages. Supports multiple browser pages and can be deployed remotely via HTTP transport.

MCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.

Enables AI agents to search and fetch high-quality information from multiple sources, including general web APIs, Wikipedia, arXiv, Hacker News, Stack Exchange, and Crossref, with optional pro-mode deep research and clean markdown page extraction.

Enables a locally-run LLM to search the web, fetch pages as markdown, make arbitrary HTTP requests, and optionally render pages with headless Chromium.

A self-hosted MCP server providing web search and URL fetching tools, running locally without external API keys or accounts.

Enables live website health checks including TLS, HTTPS, and security headers, returning an A-F grade with specific fixes.