scrape_incremental
Refresh a documentation index by fetching the sitemap, scraping new or failed pages, and re-indexing only pages with changed content.
Instructions
Incremental scrape: fetch the sitemap, scrape any new pages not previously indexed, retry previously failed pages, and re-check existing pages for content changes. Only re-indexes pages whose content has actually changed.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| concurrency | No | Number of parallel browser tabs (1-5). Default: 3. | |
| language_filter | No | Language to scrape. Same codes as scrape_full. Default: 'en'. |