"Information about crawer or crawler" matching MCP servers:

Web-crawler MCP server that fetches web pages with static parsing or headless browser fallback, enabling Claude to fetch, extract links, crawl sites, and select content via CSS selectors.

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

A comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.

Enables AI assistants to search and collect data from Xiaohongshu/REDnote, including notes, comments, and user info, with dual-layer anti-detection and persistent login.

A specialized web crawler and scraper for the US Patent and Trademark Office (USPTO) website, built with Crawl4AI and MCP integration, enabling patent and trademark search, extraction, and status checking.

This server crawls documentation sites and provides search and page-retrieval tools through MCP, enabling users to query indexed documentation content.

Enables site-wide crawling of internal links with BFS traversal, time/title/URL filtering, and page operations like summarization and link clicking, with results persisted to SQLite.

Provides live wait times and office details for all 60 Oregon DMV field offices by scraping the ODOT website, with no authentication required.

Exposes Futurepedia AI tool crawling and universal web page parsing as MCP tools for assistants.

An MCP server for crawling websites and extracting comprehensive data including images, SEO metadata, security headers, and business intelligence. It features twelve distinct extraction modes to perform detailed audits for e-commerce sites and general web analysis.

Enables AI to autonomously search, filter, and download images from the web using natural language commands, with intelligent scoring and local storage.

Enables agents to crawl URLs and retrieve page content as fixed-shape structured JSON, including title, markdown, links, word count, and error status.

A minimal MCP server that crawls web pages using crawl4ai and returns content as markdown, deployable to Cloud Run.

Enables crawling websites, including JavaScript-rendered pages, and returns extracted page content as JSON with structured error reporting.

Enables web crawling and content extraction from web pages, supporting multiple output formats like text, markdown, XML, and JSON, with robots.txt compliance and rate limiting.

Provides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.

Enables crawling news articles by category (politics, economy, society, etc.) and generates blog-ready markdown summaries. Provides MCP tools for listing categories, crawling news, getting article details, and saving to files.

An intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.

It crawls website

A single-process MCP server that crawls web pages, handles anti-bot challenges, converts HTML to Markdown, and caches results in PostgreSQL, providing tools to fetch and read web content.