discover_sitemaps
Discover active XML sitemaps for a website by reading robots.txt and testing common locations, enabling reliable URL mapping and crawl planning.
Instructions
Autonomously discovers all active XML sitemaps for a website by reading robots.txt and testing standard location heuristics (/sitemap.xml, /sitemap_index.xml, etc.).
Args: base_url: The root domain or homepage URL (e.g. 'https://stripe.com').
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| base_url | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |