Crawl a website
crawl_siteCrawl a site breadth-first from a starting URL, staying on the same domain, and return the clean text of each page. Useful for building a knowledge base or RAG corpus. Cost: 5 credits per call.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Public http(s) page URL | |
| max_pages | No | ||
| max_chars_per_page | No |