site_map
Discover all URLs on a website without scraping content. Use for sitemap audits, finding unindexed pages, and understanding site structure.
Instructions
Discover all URLs on a website without scraping their content.
Fast URL discovery that maps out a site's structure. Use for sitemap
audits (compare against sitemap.xml), finding unindexed pages, understanding
site architecture, or scoping a crawl before running it.
Args:
url: Website URL (e.g. "https://example.com")
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| config | No |