crawl
Crawl a web page to add its content to your local knowledge base. Runs in the background, returning a task ID for progress tracking.
Instructions
Crawl a web page and add it to the knowledge base (non-blocking). Launches the crawl as a background task and returns immediately with a task_id. Use crawl_status(task_id) to poll progress.
Args:
url: The URL to crawl (must start with http:// or https://).
depth: None (default) crawls the whole site; 0 fetches only this URL;
positive int caps link-follow depth.
max_pages: None (default) means no page limit. Positive int caps total
pages fetched.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| depth | No | ||
| max_pages | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||