"Web scraping and crawling tools" matching MCP servers:

Provides an LLM client with self-hosted web scraping capabilities including scraping, crawling, mapping, searching, structured data extraction, browser interaction, and deep research, without reliance on paid proxy or CAPTCHA services.

Web search, content extraction, and library docs for AI agents with 5-strategy scraping and runs without API keys.

A powerful tool for fetching and extracting text content from web pages and APIs, supporting web scraping, REST API requests, and Google Custom Search integration.

Provides browser automation and web scraping as MCP tools, enabling autonomous URL ingestion, crawling, extraction, and anti-bot handling with interactive browser control.

Built as a Model Context Protocol (MCP) server that provides advanced web search, content extraction, web crawling, and scraping capabilities using the Firecrawl API.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

Enables web scraping and crawling capabilities for LLM clients, supporting single-page scraping, multi-page website crawling, and web search with multiple engines (Playwright, Cheerio, Puppeteer) and flexible output formats including markdown, HTML, text, and screenshots.

A Model Context Protocol server that allows LLMs to interact with web content through standardized tools, currently supporting web scraping functionality.

Enables fetching, crawling, and browsing anti-bot protected web pages with real browser rendering, returning clean JSON or markdown, and solving captchas.

Enables language models to perform AI-powered web scraping, structured extraction, search, crawling, schema generation, and scheduled monitoring through an integrated web-scraping API.

Enables LLM agents to drive full web crawling and scraping through conversation by exposing tools for fetching, extracting, crawling, and screenshotting over stdio or SSE.

An MCP server providing web search, image search, and page scraping tools to LLMs without requiring API keys.

Enables deterministic calculations for AI API and subscription costs, meeting time zones, recurring schedules, and cron next-run previews, returning normalized inputs, units, assumptions, dated sources, and links to results.

Provides MCP tools for web scraping, including smart fetching, proxy rotation, and browser rendering to bypass anti-bot challenges and render JavaScript-heavy sites.

Enables AI agents to extract clean LLM-ready markdown and readable articles from web pages, parse metadata, tables, links, feeds, and contact info, audit paywalls, SEO, and XSS, and access these tools via Streamable HTTP or a stdio bridge.

Open-source browser automation API with anti-detection for AI agents, web scraping, and automation, providing undetectable Chrome via MCP and REST API with Cloudflare bypass.

Connect AI assistants to your Gramps Web family tree. Search and edit genealogy records, explore ancestors, descendants, relationships and timelines, and read document scans with vision-capable models. Includes guided research workflows, source citations, read-only mode, and privacy-aware media previews. Install with Docker or a Claude Desktop bundle.

Web search, crawling and ranking evaluation on your own YaCy peer: no API key, no central service. Results carry author-signature verdicts and declared tags.

MCP server that searches the web, reads pages, and returns concise summaries with cited sources via the research_web and read_url tools.

Free Industry 4.0 / IIoT tools, no API key or account: lint MQTT Sparkplug B topics, check a Unified Namespace (UNS) for naming and hierarchy problems, and run an industrial AI-readiness diagnostic for manufacturing digital transformation and smart-factory projects. Works in Claude Code, Codex and any MCP client.