Map a site
map_siteDiscover a website's URLs by reading its sitemaps, or fall back to scanning the start page's links when sitemaps are absent.
Instructions
List a site's addresses, from its sitemaps or its start page's links.
url: an http(s) address on the site; its links stand in when the site has no sitemap. limit: the most addresses returned, 1 to 1,000.
Returns {"ok", "url", "source", "urls", "sitemaps", "truncated"}. source is "sitemaps" or "links"; each of urls is {"url", "lastmod", "sitemap"}, only addresses on the site, in the order the sitemaps list them; sitemaps says what became of each one tried. At most ten sitemaps are read, politely, within a minute; truncated is true when a bound cut the map short, urls_left_out counting the addresses left out to keep the answer under 75,000 bytes. Hand the addresses worth reading to extract_declared, or crawl_site to follow links from one.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | an http(s) address on the site; its links stand in when the site has no sitemap. | |
| limit | No | the most addresses returned, 1 to 1,000. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ok | Yes | ||
| url | No | ||
| urls | No | ||
| error | No | ||
| source | No | ||
| sitemaps | No | ||
| truncated | No | ||
| urls_left_out | No |