Scrape URL
crw_scrapeScrape any URL to clean markdown, HTML, or links for AI agents. Handles JavaScript rendering, extracts main content, and supports custom selectors.
Instructions
Scrape one URL to markdown, HTML, or links.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| formats | No | Output formats (default ["markdown"]) | |
| waitFor | No | Ms to wait after JS render for late content | |
| renderJs | No | Force JS render (true), HTTP-only (false), omit = auto | |
| renderer | No | Pin renderer; non-auto hard-pins and implies renderJs:true (default auto). 'camoufox' requires the server's opt-in camoufox tier to be configured. | |
| maxLength | No | Max chars per content field; 0 = unbounded (default ~15000) | |
| excludeTags | No | CSS selectors to exclude | |
| includeTags | No | CSS selectors to include | |
| onlyMainContent | No | Strip nav/footer; main content only (default true) |