Enables AI agents to fetch web pages with token estimation, caching, and routing to appropriate fetchers (Trafilatura, Jina, FireCrawl, pypdf) for clean Markdown output.
Fetch web pages into clean, RAG-ready Markdown with a verifiable provenance record — source URL, UTC fetch time, SHA-256 of the response body, redirect chain, and an extraction-fidelity metric. Writes WARC archives and honours robots.txt/ai.txt, so every document an agent reads can be re-verified and cited.
Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.
Enables AI agents to reliably fetch public web URLs and extract clean, agent-ready text, Markdown, links, and metadata, with automatic browser fallback and optional x402 payment support.
Enables LLMs to retrieve and process web content by fetching URLs and converting HTML to markdown format. Supports chunked reading of large pages and can access both public websites and local networks.