crw_map
Discover website URLs by crawling pages or reading sitemap.xml to map site structure and content.
Instructions
Discover URLs on a website by crawling and/or reading its sitemap.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| maxDepth | No | Maximum crawl depth for discovery (default: 2) | |
| url | Yes | The URL to map | |
| useSitemap | No | Whether to use the site's sitemap.xml (default: true) |