Map site URLs
crw_mapMap a site's structure by discovering its URLs through sitemap.xml and a short crawl, returning a clean list of links without page content.
Instructions
Discover URLs on a site via sitemap and/or a short crawl. Returns a URL list only, no page content.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to map | |
| limit | No | Max URLs to discover AND return; 0 = unbounded (default 100). Raise it (e.g. 50000) to pull deep/large sitemaps. | |
| maxDepth | No | Max discovery depth (default 2) | |
| useSitemap | No | Use sitemap.xml (default true) | |
| crawlFallback | No | Supplement sitemap with a short BFS crawl (default true; false = sitemap-only) |