crawl
Recursively crawl websites to extract content, specifying URL, depth, and output format for efficient data collection and processing.
Instructions
Recursively crawl websites and extract content.
Input Schema
Name | Required | Description | Default |
---|---|---|---|
depth | No | Maximum depth | |
format | No | Output format | markdown |
limit | No | Maximum number of pages to crawl | |
url | Yes | Starting URL |