create_crawl_job_route
Create a crawl job to index documentation from a URL, enabling an agent to access and interact with those pages. Configure depth, page limits, sitemaps, and auto-sync.
Instructions
Create Crawl Job
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to a page of documentation that the agent will have access to in order to interact with users. | |
| pattern | No | ||
| max_depth | No | Maximum depth for crawling (1-5), defaults to 3. | |
| max_pages | No | Maximum number of pages to crawl (1-10,000), defaults to 1000. | |
| auto_remove | No | Whether to automatically remove the document if the URL becomes unavailable. Only applicable when auto-sync is enabled. | |
| sitemap_urls | No | ||
| enable_auto_sync | No | Whether to enable auto-sync for this URL document. | |
| parent_folder_id | No | ||
| minimum_frequency_days | No |