crawl_site
Start a background crawl of a configured documentation site to update its local corpus. It stages changes and only replaces the corpus on success, preserving the previous version if it fails.
Instructions
Start a background crawl for a configured site. Returns immediately with a job ID. The crawl builds in a staging area and replaces the stored corpus only when it completes, so a cancelled or failed crawl leaves the previous corpus intact.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| site_key | Yes | Site key from config file (e.g., 'langchain_py', 'rust_docs') | |
| incremental | No | Enable incremental mode (skip unchanged pages) |