"Techniques for Scraping Publicly Accessible Documents" matching MCP servers:

Safely downloads and extracts public tender documents from PDFs, Office files, and ZIP archives into text with structured signals, while also supporting document comparison and base64 extraction.

Enables AI assistants to fetch public web pages as readable text or HTML, with caching, live refresh, and parallel job handling.

Enables fetching any public web page through managed proxies, with optional JavaScript rendering, returning markdown, text, HTML, or structured JSON.

Enables running the lead scoring and outbound pipeline by talking to Claude, including scraping healthcare job postings, scoring against buyer ICP, gating on company size, and managing sequences.

Gives Claude control of a real, headed Chromium browser via Playwright, enabling web navigation, clicking, typing, screenshots, and JavaScript evaluation.

Enables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.

Automatically crawls documentation websites, converts them to organized markdown files, and generates condensed cheat sheets. Intelligently categorizes content into tools/APIs and provides local-first access to downloaded documentation.

Enables checking WCAG color contrast ratios and accessibility compliance between color pairs. Helps determine optimal text colors for backgrounds and validates color combinations meet accessibility standards.

An Operit-compatible adapter of the Exa MCP server, enabling web search, code search, and company research capabilities in AI assistants. It fixes MCP handshake compatibility issues, allowing tools like web_search_exa and web_fetch_exa to load and run reliably.

MCP server for real-time public-web lead research without search API keys, using Google Maps and web search engines with LLM-based extraction, crawling, and local lead storage/export.

Provides a local MCP server for AI-powered web crawling on Windows, enabling agents to directly crawl and extract web content without requiring WSL.

A Model Context Protocol server that provides web search capabilities using Google Custom Search API and webpage content extraction functionality.

Enables AI assistants to fetch, search, and organize menu information from For Five Coffee café. Provides access to complete menu data, category filtering, and item search capabilities through both MCP and REST API interfaces.

Enables AI agents to drive real Google Chrome with stealth and captcha handling, using Patchright to avoid bot detection.

Enables web page scraping via Jina reader API and searching FastMCP documentation using minsearch. Supports fetching markdown content from URLs and querying indexed documentation files.

Thunderbit MCP lets AI agents search, scrape, and extract structured data from any website.

MCP server for AI agents -- fetch any URL with full JavaScript rendering (Playwright/Chromium) and convert to clean, token-efficient markdown. Works on React, Vue, Angular, and any JS-heavy page. Includes web search, batch fetching, binary file download, LRU cache, SSRF protection, and structured output.

Enables searching for upcoming academic conferences and events from WikiCFP by keywords, returning detailed information including dates, locations, submission deadlines, and related resources.

MCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.

Enables MCP clients such as Claude and Cursor to fetch raw page HTML and structured data for products, real estate listings, hotels, groceries, search engine results, and public social media profiles and posts. Tools accept a URL with optional JavaScript rendering and premium proxy options, with failed requests refunded and pending social scrapes retrievable later.

⚡ 87% Token-compressed clean web scraping, YouTube AI transcripts, and PDF extraction for LLMs. Instant Free Trial with zero setup (No wallet/crypto required). Supports gasless agent vaults and EIP-712 on-chain oracle verification on Polygon, Base & Arbitrum.

Enables agents to search and read posts, images, and limited comments across Xiaohongshu, Weibo, Zhihu, and WeChat Official Accounts, with local OCR and traceable sources.

Enables AI clients to scrape anti-bot-protected and JavaScript-rendered websites, search the web, pull Wayback snapshots, transcribe YouTube videos, parse remote PDFs/Office docs, and run OSINT/GEOINT reconnaissance across 25+ platforms with confidence scoring, all backed by local SQLite caching and an OpenAPI bridge.

Enables LLMs to perform browser automation tasks including web scraping, taking screenshots, and executing JavaScript using Playwright. It facilitates real-time interaction with web pages and the generation of automated test code.

MCP server providing real-time crypto prices (Chainlink + Binance), Polymarket prediction market data, deep web research, and JS-rendered web scraping. All services available via x402 Solana micropayments.

Remote MCP / x402 / USDC / Base / Solana / OpenAPI / Google ADK / Gemini Xcatcher is an agent-first Remote MCP server (Streamable HTTP) for high-throughput crawling of fresh/latest posts across large sets of X (Twitter) usernames. It supports x402 pay-as-you-go top-ups using USDC on Base and Solana.

Enables AI agents to extract clean LLM-ready markdown and readable articles from web pages, parse metadata, tables, links, feeds, and contact info, audit paywalls, SEO, and XSS, and access these tools via Streamable HTTP or a stdio bridge.

MCP server for searching Fiverr gigs, comparing freelancer packages, reading seller reviews, and exploring service categories. Provides structured tools for gig search, details, seller profiles, reviews, and category browsing with built-in anti-bot handling.

TypeScript implementation of an MCP server that provides tools for web crawling, content extraction, and browser automation, enabling AI systems to access and process web content through 15 specialized tools.

Enables crawling sites from start URLs and retrieving internal and external links, file download links, HTML tables as rows, and lists, with controls for crawl depth, page limits, sitemaps, robots.txt, and proxy mode.