crawl_sitemap
Stream and parse XML sitemaps or .xml.gz files to extract URLs, following nested child sitemaps in sitemap indexes so you can map a site's pages quickly.
Instructions
High-Speed XML Sitemap & Index Parser: Streams and parses XML sitemaps, handling namespaces, .xml.gz gzip streams, and automatically crawls child sitemaps if a sitemap index is provided.
Args: sitemap_url: URL of the sitemap (e.g. 'https://example.com/sitemap.xml'). max_urls: Maximum URLs to extract (default 200). crawl_nested: If true, automatically follows nested sitemaps inside sitemapindex.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| max_urls | No | ||
| sitemap_url | Yes | ||
| crawl_nested | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |