Scrape URL
crw_scrapeScrape any URL to markdown, HTML, or links. Customize with wait times, JS rendering, and CSS selectors to include or exclude content.
Instructions
Scrape one URL to markdown, HTML, or links.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| formats | No | Output formats (default ["markdown"]) | |
| waitFor | No | Ms to wait after JS render for late content | |
| renderJs | No | Force JS render (true), HTTP-only (false), omit = auto | |
| renderer | No | Pin renderer; non-auto hard-pins and implies renderJs:true (default auto). 'camoufox' requires the server's opt-in camoufox tier to be configured. | |
| maxLength | No | Max chars per content field; 0 = unbounded (default ~15000) | |
| excludeTags | No | CSS selectors to exclude | |
| includeTags | No | CSS selectors to include | |
| onlyMainContent | No | Strip nav/footer; main content only (default true) |