"Information about web crawlers or scraping techniques" matching MCP servers:

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

An MCP server that enables AI assistants to perform web searches on Perplexity.ai using browser automation instead of an official API. It supports persistent authenticated sessions and returns search results along with cited sources directly to the client.

Provides programmatic access to Material Web documentation and components for AI agents, enabling documentation search, template generation, and HTML validation.

A Python MCP server for web research that enables fetching web pages, searching, batch-fetching, extracting links, and summarizing content via client-side sampling, with optional local file access.

Enables AI assistants to access real-time web data through search, markdown scraping, and browser automation while bypassing anti-bot protections. It provides tools for web research, e-commerce monitoring, and data extraction from across the globe.

Provides AI assistants with access to ShopSavvy's comprehensive product data, pricing information, and historical price tracking, enabling product lookup by barcode or ASIN, current offers, price history, and scheduling for automatic monitoring.

A Firecrawl MCP server with built-in support for multiple API key rotation, enabling web search, scraping, crawling, extraction, and more via MCP clients.

A lightweight MCP server that enables AI assistants to execute local development tools and retrieve system status with low latency over stdio or HTTP.

Live Japan seasonal travel data from Japan Meteorological Corporation. 12 tools covering cherry blossom (sakura) forecasts, autumn leaves (koyo), fruit picking, flowers, festivals, and 3-day weather. 1,700+ GPS-tagged spots updated daily at 9AM JST. Works via npx or hosted HTTP endpoint — no auth required.

Enables interaction with Steam: search games, get store details, reviews, prices, discounts, news, and player profiles, libraries, and achievements via the Steam Web API.

Standalone MCP server for Obsidian vaults - hybrid search (FTS5 + vector + cross-encoder reranking), images and PDFs in agent-readable form, Kanban-aware tasks (Tasks-plugin + Dataview formats), structured memory with topic recall, fine-grained read/write tools for optimal token efficiency, and link graph support. Run locally, self-host, or one-click deploy for remote access. OAuth 2.1.

Enables any MCP-compatible LLM to perform AI-powered web research using Google's AI Mode, returning synthesized and cited answers instead of raw search results.

Fetches YouTube video transcripts, using free captions when available and falling back to Gemini-based transcription for missing or broken captions.

The MCP server for Nanostores that scans your codebase with AST analysis to find stores, subscribers, and dependency chains, and can ingest live events to spot noisy, error-prone, or unused stores.

Thin MCP wrapper around the SQD Portal API for blockchain data queries across multiple networks including EVM, Solana, Bitcoin, Substrate, and Hyperliquid. Provides 25 public tools for discovery, cross-chain queries, and chain-specific operations.

Enables MCP-compatible assistants to lint, format, render, parse, and migrate Carve documents directly from source text, without filesystem or network access.

The interface protocol for AI agents. 8 kernel primitives + 16 stdlib operations to operate any interface. Forge once, run forever — zero AI at runtime. 81 skills across 41 sites.

MCP server for the Oura Ring API v2, enabling natural language queries about sleep, readiness, activity, heart rate, and more.

Integrates HomeMatic smart home devices (via CCU3, RaspberryMatic, or debmatic) as MCP tools, enabling natural language query and control of rooms, devices, programs, and system variables.

Provides AI coding assistants with access to AccelByte Gaming Services (AGS) Extend SDK functions and models through searchable symbols and detailed documentation. Helps generate SDK code by exposing context about available functions, models, and templates for creating Extend app projects across multiple languages (Go, C#, Java, Python).