getOgScrapeData
Scrapes raw HTML of a webpage for custom parsing, link extraction, or DOM inspection. Supports JavaScript rendering for SPA pages.
Instructions
Scrape and return the raw HTML of a URL via the OpenGraph.io API (v3). Returns the complete page HTML — use this when you need to do your own parsing, extract all links, inspect the DOM structure, or feed raw markup into another tool or model. The text response includes the first 3 000 characters; the full HTML is in the structured html field.
For JS-heavy or single-page applications set full_render: true to guarantee JavaScript execution before the HTML is captured. For most sites, the default auto_render handles this automatically.
Pick the right tool: getOgData → Open Graph tags, social preview metadata (title, description, image, favicon) getOgMarkdown → Clean readable text / article prose — ideal for feeding into an LLM getOgScrapeData → Raw HTML — use when you need to do your own parsing or link extraction getOgExtract → Targeted elements by tag (html_elements) or named CSS selectors (selectors) getOgScreenshot → Visual capture of a page as an image getOgQuery → Natural-language question answered from page content (100–200 credits/request)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the webpage to scrape. | |
| retry | No | Automatically retry failed requests with escalating proxy tiers. Defaults to true on v3. | |
| cache_ok | No | Use cached results. Set to false to bypass cache and get fresh data. Defaults to true. | |
| use_proxy | No | Route the request through a standard proxy. | |
| auto_proxy | No | Automatically escalate to a proxy if the direct request fails. Defaults to true on v3. | |
| accept_lang | No | Accept-Language header for the outbound request. Use 'auto' to mirror the caller's language. Defaults to 'auto'. | |
| ai_sanitize | No | Scan the fetched content for prompt-injection attempts before returning it. | |
| auto_render | No | Automatically detects JS-heavy / SPA pages and re-fetches with browser rendering when needed. Enabled by default on v3 — leave unset unless you want to disable it. For guaranteed JS execution on every request use full_render: true instead. | |
| full_render | No | Forces a full browser execution pass on every request regardless of page type. Use when auto_render hasn't produced the content you expected, or when you need guaranteed JavaScript execution. Slower than auto_render — prefer auto_render for most cases. | |
| max_retries | No | Maximum number of retry attempts (1–4). Defaults to 4. | |
| use_premium | No | Route the request through a premium proxy. | |
| use_superior | No | Route the request through a superior-tier proxy. | |
| max_cache_age | No | Maximum cache age in milliseconds. Results older than this will be re-fetched. Defaults to 432000000 (5 days). | |
| proxy_country | No | Two-letter ISO country code for geo-targeted proxy exit node (e.g. 'US', 'GB'). | |
| load_more_wait | No | Milliseconds to wait after each load_more click (0–5000). Defaults to 1500. | |
| retry_escalate | No | Escalate proxy tier on each retry attempt. Defaults to true. | |
| ai_sanitize_mode | No | 'sanitize' cleans the content, 'warn' returns it with a safety report, 'block' returns HTTP 422 when risk_score >= 0.7. | |
| load_more_clicks | No | Number of times to click the load_more_selector (1–10). Defaults to 3. | |
| load_more_scroll | No | Scroll between load_more clicks. Defaults to true. | |
| scroll_to_bottom | No | Scroll to the bottom of the page before scraping. Useful for lazy-loaded content. | |
| wait_for_selector | No | CSS selector to wait for before scraping. Forces full_render. | |
| load_more_selector | No | CSS selector for a 'load more' button to click before scraping. | |
| load_more_item_selector | No | CSS selector to watch for new items when using load_more_selector. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| html | Yes | Full raw HTML of the scraped page | |
| length | Yes | Character count of the HTML | |
| requestInfo | No | Request metadata (cache status, version, options echo) |