scrape
Scrape any public webpage by providing a target URL and API key. Automatically handles rotating proxies, CAPTCHA bypass, and optional JavaScript rendering to return the raw HTML.
Instructions
Scrapes any public webpage. Pass your API key and target URL; Scrapingdog handles rotating proxies, CAPTCHA bypass, and optional JavaScript rendering automatically. Returns the raw HTML of the target page. [Credits: 1 credit (dynamic=false, no premium). 5 credits with JS rendering (dynamic=true, the default). 10 credits with premium=true (residential proxies). 25 credits when premium=true and dynamic=true are combined. 10 credits when stealth_mode=true.] Notes: All feature parameters above (dynamic, premium, country, session_number, stealth_mode, custom_headers, wait) are documented on separate sub-pages of the Web Scraping API doc but apply to this single GET /scrape endpoint — they are not separate tools/endpoints. Custom headers: attach real HTTP headers to your request to Scrapingdog (not as a query param value) and also set custom_headers=true; Scrapingdog relays them to the target site. Geotargeting (country) combines with premium for non-US residential exit nodes. Sessions (session_number) keep the same proxy IP for up to 60s of inactivity, useful for multi-step flows (e.g. login then scrape). Stealth Mode (stealth_mode=true) is the CAPTCHA/Cloudflare-bypass mechanism — there is no separate 'captcha' parameter documented. No markdown-output parameter or ai_query parameter was found documented on any of the fetched pages for this endpoint (unlike some other Scrapingdog APIs) — response is raw HTML only. Returns: Raw HTML of the target page (Content-Type text/html), returned as the response body exactly as fetched/rendered by Scrapingdog's proxy/browser infrastructure.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The target URL to scrape. Should be URL-encoded. | |
| wait | No | Gives slow-loading dynamic pages extra time to finish loading before the page is scraped. Mentioned in the Javascript Rendering doc's prose only — no worked example or explicit unit/range was shown; commonly milliseconds in comparable APIs, but this is not confirmed by the docs. | |
| country | No | ISO 3166-1 country code (e.g. 'us') controlling which country the proxy appears to be from. Mainly useful for search engines / geo-sensitive sites. Available on all plans from Free to Enterprise. | |
| dynamic | No | Enables JavaScript rendering via headless Chrome for JS-heavy pages. Enabled by default, raising cost to 5 credits (25 with premium). Set dynamic=false to scrape static HTML only and reduce cost to 1 credit when JS rendering is not needed. (default: true) | |
| premium | No | Enables premium residential proxies for websites that are difficult to scrape with standard datacenter proxies. Defaults to USA-based proxies but supports geotargeting via the country parameter. Costs 10 credits per request, or 25 credits when combined with dynamic=true (JS rendering). (default: false) | |
| stealth_mode | No | Stealth Mode bypasses Cloudflare and similar bot-protection/CAPTCHA systems using advanced browser fingerprinting techniques. Costs 10 credits per request. (default: false) | |
| custom_headers | No | Set to true to have Scrapingdog forward custom HTTP headers you attach to the request (e.g. via a header like 'X-customheader') on to the target site. Useful for authentication-protected content or sites requiring specific headers. No additional cost. (default: false) | |
| session_number | No | Reuses the same proxy IP across multiple scraping requests. Any integer value creates or resumes a session; use the same number across requests to keep the same proxy. Sessions automatically expire 60 seconds after last use. No additional cost. |