Enables AI agents to fetch web pages with token estimation, caching, and routing to appropriate fetchers (Trafilatura, Jina, FireCrawl, pypdf) for clean Markdown output.
Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.
Enables LLMs to fetch and process web content by converting HTML into markdown for easier consumption. It supports chunked reading via pagination and provides configuration options for robots.txt compliance and proxy usage.