openapi_v2_webtools_scrape
Scrape a single URL
Scrape a single URL and return clean content.
Specify one or more of markdown / json / rawHtml in
formats. json returns a structured page summary
(title, page_type, key facts, main entities, etc.) — the extraction
schema is built in, so you don't need to define one in your request.
Response meta carries the target page's HTTP status code, title,
canonical URL, description, and detected language. Always check
meta.statusCode before trusting the content body: a 4xx/5xx (e.g. 404
or a 5xx) usually means the content is an error page, not the real page.
A page that refuses the request (HTTP 401/403/451/503, or one the
upstream flags as blocked) returns success:false with an error.code
of ACCESS_DENIED plus a customer-facing error.message directing
the caller to support — these domains stay refused under retry, so the
response is actionable rather than transient. A 429 returns RATE_LIMITED
with a generic retry message; UNREACHABLE / TIMEOUT /
CONTENT_UNAVAILABLE cover host/network/extraction failures. The
error.details payload is reserved for future structured attribution and
is currently always null; callers should branch on error.code and
surface error.message to end users. Refused requests are not billed
and never carry a content body.
Responses:
200: Successful Response (Success Response) Content-Type: application/json
Example Response:
{
"success": true,
"meta": {
"requestId": "Requestid",
"timestamp": "Timestamp"
}
}Output Schema:
{
"properties": {
"success": {
"type": "boolean",
"title": "Success",
"description": "Whether the request was successful",
"default": true
},
"data": {
"description": "Response data payload"
},
"error": {
"description": "Error details if request failed"
},
"meta": {
"description": "Metadata for API responses.\n\nCredit fields follow the ADR-0003 parallel-fields strategy (Option 3):\n- `credits_remaining` / `credits_consumed` (int): legacy fields, rounded\n to whole credits, kept for zero-breaking-change to existing SDK clients.\n- `credits_remaining_exact` / `credits_consumed_exact` (float): new\n precision-aware fields for clients that opt in to decimal credits.\n\nSee ADR-0003 decision 5 and the \u00a78 deprecation timeline.\n\nTODO(2026-11, ADR-0003 \u00a78 +6mo): mark `credits_remaining` /\n`credits_consumed` as `deprecated=True` in their Field() definitions\nand announce in customer changelog.\nTODO(2027-05, ADR-0003 \u00a78 +12mo): remove the legacy int fields via a\nmajor-version bump of the OpenAPI surface.",
"properties": {
"requestId": {
"type": "string",
"title": "Requestid",
"description": "Unique request identifier"
},
"timestamp": {
"type": "string",
"title": "Timestamp",
"description": "Response timestamp in ISO 8601 format"
},
"total": {
"title": "Total",
"description": "Total number of records"
},
"page": {
"title": "Page",
"description": "Current page number"
},
"pageSize": {
"title": "Pagesize",
"description": "Number of records per page"
},
"totalPages": {
"title": "Totalpages",
"description": "Total number of pages"
},
"creditsRemaining": {
"title": "Creditsremaining",
"description": "Remaining API credits (rounded to whole credits; see creditsRemainingExact for precise value)"
},
"creditsConsumed": {
"title": "Creditsconsumed",
"description": "Credits consumed by this request (rounded; see creditsConsumedExact for precise value)"
},
"creditsRemainingExact": {
"title": "Creditsremainingexact",
"description": "Remaining API credits, precise to 1 decimal place"
},
"creditsConsumedExact": {
"title": "Creditsconsumedexact",
"description": "Credits consumed by this request, precise to 1 decimal place"
},
"tokensUsage": {
"description": "Provider token-usage block \u2014 populated on terminal video polls only, null on every non-video endpoint. See TokensUsage for its fields."
}
},
"type": "object",
"required": [
"requestId",
"timestamp"
],
"title": "ResponseMeta"
}
},
"type": "object",
"required": [
"meta"
],
"title": "OpenApiResponse[CrawlerScrape]",
"examples": []
}422: Validation Error Content-Type: application/json
Example Response:
{
"detail": [
{
"loc": [],
"msg": "Message",
"type": "Error Type",
"ctx": {}
}
]
}Output Schema:
{
"properties": {
"detail": {
"items": {
"properties": {
"loc": {
"items": {},
"type": "array",
"title": "Location"
},
"msg": {
"type": "string",
"title": "Message"
},
"type": {
"type": "string",
"title": "Error Type"
},
"input": {
"title": "Input"
},
"ctx": {
"type": "object",
"title": "Context"
}
},
"type": "object",
"required": [
"loc",
"msg",
"type"
],
"title": "ValidationError"
},
"type": "array",
"title": "Detail"
}
},
"type": "object",
"title": "HTTPValidationError"
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| formats | No | Output formats. Order in the response matches the input order. |