Initiate a structured web crawl from a specified URL, controlling depth, breadth, and focus on specific sections or domains using regex and predefined categories. Extract content in markdown or text format for targeted data retrieval.
Extract and analyze web page content to answer specific questions using RAG (Retrieval Augmented Generation). Provide AI-generated responses based on relevant page sections for accurate insights.
An intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.
Enables fetching and reading content from web URLs through a configurable proxy prefix. Uses curl to retrieve webpage content and return it as text for analysis or processing.
An MCP server for crawling websites and extracting comprehensive data including images, SEO metadata, security headers, and business intelligence. It features twelve distinct extraction modes to perform detailed audits for e-commerce sites and general web analysis.