crawl_website
Crawl a website from a specified URL, extracting content up to a defined depth and page limit. Returns structured data for crawled pages.
Instructions
Crawl a website starting from the given URL up to a specified depth and page limit.
Args: url: The starting URL to crawl. crawl_depth: The maximum depth to crawl relative to the starting URL (default: 1). max_pages: The maximum number of pages to scrape during the crawl (default: 5).
Returns: List containing TextContent with a JSON array of results for crawled pages.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| crawl_depth | No | ||
| max_pages | No | ||
| url | Yes |
Input Schema (JSON Schema)
{
"properties": {
"crawl_depth": {
"default": 1,
"title": "Crawl Depth",
"type": "integer"
},
"max_pages": {
"default": 5,
"title": "Max Pages",
"type": "integer"
},
"url": {
"title": "Url",
"type": "string"
}
},
"required": [
"url"
],
"title": "crawl_websiteArguments",
"type": "object"
}