"Web scraping and crawling tools" matching MCP servers:

Enables fetching any public web page through managed proxies, with optional JavaScript rendering, returning markdown, text, HTML, or structured JSON.

A powerful tool for fetching and extracting text content from web pages and APIs, supporting web scraping, REST API requests, and Google Custom Search integration.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

ZeroCrawl MCP is a 100% free web crawling and scraping server for AI agents, offering tools like page scraping, domain crawling, metadata extraction, screenshots, sitemap parsing, batch scraping, and search-and-crawl. It requires no API keys and integrates with Claude Desktop, Cursor, and other MCP clients.

Provides 63 production-ready API tools for AI agents, including web scraping, AI generation, crypto data, OCR, image generation, audio transcription, text-to-speech, email, and domain lookup.

Provides MCP tools for web scraping, including smart fetching, proxy rotation, and browser rendering to bypass anti-bot challenges and render JavaScript-heavy sites.

Provides web access tools (fetch, search, sitemap crawl) through String AI's API, automatically handling anti-bot bypass, CAPTCHA, and JavaScript rendering.

Browser automation that bypasses anti-bot systems and handles web-scraping with CDP Mode.

MCP server for SrezAI providing web search, page reading, screenshots, and schema-based extraction as native agent tools.

Enables deterministic calculations for AI API and subscription costs, meeting time zones, recurring schedules, and cron next-run previews, returning normalized inputs, units, assumptions, dated sources, and links to results.

Connect AI assistants to your Gramps Web family tree. Search and edit genealogy records, explore ancestors, descendants, relationships and timelines, and read document scans with vision-capable models. Includes guided research workflows, source citations, read-only mode, and privacy-aware media previews. Install with Docker or a Claude Desktop bundle.

EU-native web scraping for AI agents & developers. scrape web pages into markdown, screenshots, metadata, and links, map sites, run asynchronous scrape jobs, and check usage, with EU-hosted, GDPR-aligned processing.

pay-per-call x402 API endpoints as MCP tools — web search, page-to-markdown, PDF extract, JSON repair/validate, crypto price, weather, IP geo, DNS, markdown lint + 20 agent-commerce aliases. Each tool proxies to vibes-coded.com and settles via x402 (USDC/Solana). Free-trial endpoints need no auth; paid ones need an x402 signature. Catalog is fetched live, so new endpoints appear automatically.

Ad-free web scraping and search exposed as 3 MCP tools. fetch_page, fetch_pages_batch, search_web. Works with Claude Desktop, Cursor, Cline.

Web search, crawling and ranking evaluation on your own YaCy peer: no API key, no central service. Results carry author-signature verdicts and declared tags.

Official MCP server for Safe Mind, the German platform for the legally required psychological risk assessment (Gefährdungsbeurteilung psychischer Belastung). 12 tools: account onboarding, departments and locations, survey drafts, and anonymity-protected aggregated analytics that never return groups below the minimum size.

Free Industry 4.0 / IIoT tools, no API key or account: lint MQTT Sparkplug B topics, check a Unified Namespace (UNS) for naming and hierarchy problems, and run an industrial AI-readiness diagnostic for manufacturing digital transformation and smart-factory projects. Works in Claude Code, Codex and any MCP client.

An MCP server providing web search, image search, and page scraping tools to LLMs without requiring API keys.

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.