crawl
Recursively crawl websites to extract content in markdown, text, or raw formats. Specify URL, depth, and page limit for tailored web data collection.
Instructions
Recursively crawl websites and extract content.
Input Schema
Name | Required | Description | Default |
---|---|---|---|
depth | No | Maximum depth | |
format | No | Output format | markdown |
limit | No | Maximum number of pages to crawl | |
url | Yes | Starting URL |