Map site URLs
crw_mapDiscover URLs on a website by parsing its sitemap and optionally performing a short BFS crawl. Returns a list of URLs without page content.
Instructions
Discover URLs on a site via sitemap and/or a short crawl. Returns a URL list only, no page content.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to map | |
| limit | No | Max URLs returned; 0 = unbounded (default 100) | |
| maxDepth | No | Max discovery depth (default 2) | |
| useSitemap | No | Use sitemap.xml (default true) | |
| crawlFallback | No | Supplement sitemap with a short BFS crawl (default true; false = sitemap-only) |