Check a crawl and read its URLs
web_access_sitemap_statusPoll a sitemap crawl job until it finishes and get the discovered URLs, with paging to retrieve each batch of found pages.
Instructions
Poll a crawl job until it is terminal; the terminal response also carries the URLs it discovered. Statuses: awaiting_approval → running → terminal completed | failed | canceled | token_cap_exceeded (budget hit before maxPages; collected results are still readable). While running it returns pending and processed counts and no URLs; a partial_state status means an interrupted start — call web_access_sitemap_start again to repair it. Poll every few seconds for small crawls; give hundreds-of-pages crawls tens of seconds between polls.
Once the job is terminal, the response carries total and urls, one page of the discovered URLs. Optional limit (1–5000, default 1000) and offset (≥ 0) choose the page; out-of-range values are rejected. Each entry has url, statusCode (0 = discovered but not fetched), depth, parentUrl, isSitemap, sourceType, and an error when that page failed. discoveredUrls (links found on the page) is only present for ~1h after completion; afterwards results come from durable storage which omits it — everything else stays available.
Requires jobId. A 404 means the job doesn't exist or belongs to another account. If the job is terminal but its URLs cannot be read, the call fails rather than return the status without them; call it again.
Returns: the jobId and status, with pending, processed, pagesProcessed, estimatedCostUsd, errorMessage and finishedAt as the job reports them, and total and urls once it is terminal.
Next: call again with offset advanced by the page size until it reaches total; read a page's content with web_access_fetch.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"jobId": "<jobId>"
},
"name": "web_access_sitemap_status"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The jobId web_access_sitemap_quote returned. | |
| limit | No | Once the job is terminal: URLs per page, 1-5000 (default 1000). | |
| offset | No | Once the job is terminal: number of URLs to skip for pagination (0 or more). |