fetch_sitemap
Fetch a sitemap.xml or sitemap index and extract listed URLs with metadata like lastmod, changefreq, and priority. Handles nested sitemap indexes, gzipped files, and partial failures without aborting the request.
Instructions
Fetch a sitemap.xml (or sitemap-index) and return the contained URLs with their lastmod / changefreq / priority. Follows sitemap-index chaining up to max_depth levels. Gzipped .xml.gz payloads are auto-decompressed. Partial failures (one child sitemap 500s while others work) are returned under 'warnings' without aborting the whole request. SSRF-protected by default.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Sitemap URL (sitemap.xml, sitemap.xml.gz, or a sitemap index) | |
| max_urls | No | Cap on total URLs returned (default 5000) | |
| max_bytes | No | Max bytes to read per sitemap response (default 20MiB) | |
| max_depth | No | How many sitemap-index levels to follow (default 1). 0 keeps the top-level index flat and only returns its childSitemaps list. | |
| timeout_ms | No | ||
| user_agent | No | ||
| max_redirects | No | ||
| allow_private_hosts | No |