Enables web content scanning and analysis by fetching, analyzing, and extracting information from web pages using tools like page fetching, link extraction, site crawling, and more.
Enables web crawling and content extraction from web pages, supporting multiple output formats like text, markdown, XML, and JSON, with robots.txt compliance and rate limiting.
Enables querying and retrieving content from webpages by parsing sitemap.xml files and fetching HTML content from specified URLs. Includes built-in rate limiting for protection against abuse.