Map a site's declared URLs
map_siteList every URL a site declares in its sitemaps -- found through robots.txt and the well-known paths, each index file walked to its children -- without fetching any of the pages. Use it first, to see how big a site is and what sections it has, before crawl_site or create_project; it cannot read page content (scrape_urls or crawl_site do) and misses pages a site links to but does not declare. Costs 1 credit per sitemap file read, usually 1 in total, never per URL. Returns the discovery method, totals, up to limit URLs and the section names.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Any URL on the site, usually its home page. | |
| limit | No | The most URLs to return, 1 to 5,000 (default 1,000); totals still count them all. | |
| search | No | Keep only URLs containing this text, e.g. '/blog/'; omit for all. |