scrape_with_js
Scrape JavaScript-rendered pages by automating a headless browser, enabling extraction of dynamic content with options for proxies, screenshots, and custom load waits.
Instructions
Scrape JavaScript-rendered pages using headless browser
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| body | No | Request body for POST/PUT | |
| super | No | Use residential & mobile proxy network (Super requests) | |
| width | No | Browser viewport width (default: 1920) | |
| device | No | Device type to emulate | |
| height | No | Browser viewport height (default: 1080) | |
| method | No | HTTP method (default: GET) | |
| output | No | Output format (raw HTML/text or markdown) | |
| render | No | Enable JavaScript rendering (forced true) | |
| geoCode | No | Country code for proxy location (defaults to "us" when Super and region unspecified) | |
| timeout | No | Request timeout in milliseconds (5000-120000) | |
| blockAds | No | Block advertisements | |
| callback | No | Webhook URL for asynchronous delivery (url-encoded) | |
| sessionId | No | Sticky session ID (0-1000000) for maintaining the same IP | |
| waitUntil | No | Wait condition for page load | |
| customWait | No | Additional wait time in milliseconds | |
| returnJSON | No | Return Scrape.do JSON payload (required for screenshots/frames data) | |
| screenShot | No | Capture default viewport screenshot (requires render & returnJSON) | |
| setCookies | No | Send cookies to the target site (use JSON string or cookie header) | |
| showFrames | No | Return iframe/frame metadata (requires returnJSON) | |
| pureCookies | No | Return cookies exactly as sent by the target | |
| disableRetry | No | Disable automatic retry on failure | |
| extraHeaders | No | Forward extra upstream headers | |
| retryTimeout | No | Retry timeout in milliseconds (5000-55000) | |
| waitSelector | No | CSS selector to wait for before capturing | |
| customHeaders | No | Let Scrape.do add default headers automatically | |
| blockResources | No | Block images, CSS, fonts to speed up loading | |
| forwardHeaders | No | Forward client headers to the target site | |
| fullScreenShot | No | Capture full-page screenshot (requires render & returnJSON) | |
| playWithBrowser | No | JSON-encoded Play-with-Browser action list (array or DSL string) | |
| regionalGeoCode | No | Regional proxy location | |
| disableRedirection | No | Disable following redirects | |
| transparentResponse | No | Return the origin response body directly with no parsing | |
| particularScreenShot | No | Capture CSS selector screenshot (selector must be URL-encoded) | |
| showWebsocketRequests | No | Return websocket request/response logs (requires returnJSON) |