Read a web page
scrapeRead one public web page and return its content: the main text (as Markdown with response_format="markdown"), links, metadata and a quality score, plus the raw HTML if you ask (return_html). Set render=true for pages that build their content with JavaScript, or mode="auto" to render only when a plain fetch comes back empty. extract pulls named fields out with CSS selectors. Costs 1 credit for a plain fetch and 5 credits when the page is rendered; a failed call costs nothing. Pages that disallow automated access or need a login cannot be read.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The full public http(s) address of the page. | |
| mode | No | "manual" (the default) does what render says. "auto" fetches the page plainly and renders it only when it looks empty; it holds up to max_cost credits and charges what it used. | manual |
| render | No | Run the page's JavaScript in a browser first, for pages that build their content in the browser. Costs more. | |
| country | No | Two-letter country code for the language headers of the request, for example ae or gb. It does not route the request through that country. | |
| extract | No | Named fields to pull out with a CSS selector, for example {"heading":{"selector":"h1"},"links":{"selector":"a","attribute":"href", "multiple":true}}. One tag, class, ID or attribute selector each. Static pages only. | |
| wait_ms | No | Milliseconds to wait after the page loads. Needs render=true. | |
| max_cost | No | With mode=auto, the most to spend: 1 never renders, 5 may render. | |
| wait_for | No | A CSS selector to wait for before reading. Needs render=true. | |
| return_html | No | Also return the page's raw HTML in data.html (up to 2 MB). Not with extract. | |
| idempotency_key | No | Optional. Send the same key (16 to 128 letters, digits, _ . : -) to retry this exact call safely: a retry within 24 hours returns the first result and is not charged again. | |
| response_format | No | Use "markdown" to get the page's main content as Markdown in data.markdown, the best form for reading or summarising. "all" (the default) and "text" return the standard fields: text, links, metadata and a quality score. | all |
| min_quality_score | No | Refuse the page, at no charge, when its quality score is below this. |