webextrator_extract
Extract structured content from URLs: product data, articles, or general page details, using JavaScript rendering and optional LLM normalization. Supports sync and async execution.
Instructions
Extract structured content from a web page using the WebExtrator API.
Navigates to the specified URL, renders the page, and extracts structured data
such as product details, article content, or general page information.
Use this when:
- You need to extract structured data from a web page
- You want product details, article content, or general page data
- You need LLM-enhanced semantic normalization of extracted content
Returns:
By default (async) a JSON envelope with a `task_id` — poll
`webextrator_get_task` until it reports `finished_at`, then read the
extracted content from its `response`. With mode="sync", the extracted
structured content inline.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the web page to extract content from. Required. | |
| mode | No | Processing mode. Defaults to 'async': returns immediately with a task_id to poll via webextrator_get_task. Pass 'sync' to wait for the result inline (may time out on slow pages). | |
| delay | No | Extra delay in seconds after page load before extracting. | |
| cookies | No | Cookies to install before navigation. Each cookie is an object with at least 'name' and 'value', plus optional 'domain', 'path', 'expires', 'httpOnly', 'secure', 'sameSite'. Useful for authenticated pages. | |
| headers | No | Extra HTTP headers to include with the page request. | |
| timeout | No | Total timeout in seconds for page load. Default is 30. | |
| enable_llm | No | Enable LLM-based semantic normalization for richer structured output. Default is false. | |
| user_agent | No | Override the User-Agent header for the page request. | |
| wait_until | No | Page load wait condition before extracting. Options: 'load', 'domcontentloaded', 'networkidle', 'commit'. Default is 'networkidle'. | |
| bypass_cache | No | Skip the Redis result cache for this request (still writes the fresh result back). Default is false. | |
| callback_url | No | Callback URL for async processing. If provided, the task runs asynchronously and results are sent to this URL when complete. | |
| expected_type | No | Hint about expected page type. Options: 'product', 'article', 'general'. Helps the extractor optimize for the content structure. | |
| block_resources | No | Resource types to block during page load to speed up rendering. Options: 'image', 'font', 'media', 'stylesheet', 'xhr', 'fetch'. | |
| cache_ttl_seconds | No | Override the global cache TTL (seconds) for this entry. 0 means do not cache this response. Default is 3600. | |
| wait_for_selector | No | CSS selector to wait for before extracting content. |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |