Get Structured Site Data
webscraping_ai_dataExtract structured JSON from any public web page by its normal URL, including parse status and consistent snake_case fields. Use it to turn pages into reusable data without writing site-specific parsers.
Instructions
Get structured JSON for any public web page from its normal URL. Returns {request_parameters: {url, provider, type}, parse_status, data}: provider is the site and type the kind of page, parse_status is ok, parse_failed or not_found, and data holds snake_case fields whose names stay the same for every page of that kind on that site (null for values the page doesn't expose; data itself can be null when parsing fails). Some sites have dedicated parsers; for others, the first request for a new kind of page also learns its structure and can take up to about 45 seconds. When learning takes longer, it returns an error result with status_code 503 and body.error_code "structure_processing" (not charged): retry after body.retry_after seconds. A 422 "structure_unavailable" means no structured data was found (not charged); use webscraping_ai_fields to name the values you want instead. On those sites, redirects to another site aren't followed: they return a 500 with body.error_code "target_redirect" naming the target (not charged). Priced per request (see https://webscraping.ai/docs#data), including parse_failed and not_found results; failed fetches are not charged.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Normal URL of any public web page, e.g. https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html. Sent as-is; the site and page type are detected from it. | |
| params | No | Extra site-specific query parameters sent to the API as-is, as an object of string, number or boolean values, for parameters added after this tool was released. Must not contain url, api_key or the parameters above. | |
| country | No | Two-letter country code of the proxy used to fetch the page, e.g. us, gb, de (us by default). | |
| transcript | No | Video pages on sites with a dedicated parser only. Also fetch the video's transcript into data.transcript (null when no matching captions are available; false by default). If the transcript fetch fails, the whole request fails with a 500 and is not charged. | |
| transcript_language | No | With transcript: true. Caption language to pick, e.g. en, de. Without it, English is preferred, then the first available track; if the video has no captions in that language, data.transcript is null. | |
| disable_content_sandboxing | No | Return the raw result without the external-content security boundaries that guard against prompt injection (false by default). |