map
Extract website URLs by analyzing sitemaps and robots.txt files to identify content structure and available pages.
Instructions
Discover URLs from a website's sitemaps (robots.txt + sitemap.xml).
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Base URL to discover sitemaps from (e.g. `<https://example.com>`) |