webextrator_extract
Extract structured content from any web page by rendering it and returning product details, article content, or general page information, with optional LLM-enhanced semantic normalization.
Instructions
Extract structured content from a web page using the WebExtrator API.
Navigates to the specified URL, renders the page, and extracts structured data
such as product details, article content, or general page information.
Use this when:
- You need to extract structured data from a web page
- You want product details, article content, or general page data
- You need LLM-enhanced semantic normalization of extracted content
Returns:
By default, the extracted structured content inline. With mode="async", a JSON envelope with a `task_id` — poll `webextrator_get_task` until it reports `finished_at`, then read the extracted content from its `response`.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the web page to extract content from. Required. | |
| mode | No | Processing mode. Defaults to API sync behavior. Pass 'async' to return immediately with a task_id to poll via webextrator_get_task. | |
| delay | No | Extra delay in seconds after page load before extracting. | |
| cookies | No | Cookies to install before navigation. Each cookie is an object with at least 'name' and 'value', plus optional 'domain', 'path', 'expires', 'httpOnly', 'secure', 'sameSite'. Useful for authenticated pages. | |
| headers | No | Extra HTTP headers to include with the page request. | |
| timeout | No | Total timeout in seconds for page load. Default is 30. | |
| enable_llm | No | Enable LLM-based semantic normalization for richer structured output. Default is false. | |
| user_agent | No | Override the User-Agent header for the page request. | |
| wait_until | No | Page load wait condition before extracting. Options: 'load', 'domcontentloaded', 'networkidle', 'commit'. Default is 'networkidle'. | |
| bypass_cache | No | Skip the Redis result cache for this request (still writes the fresh result back). Default is false. | |
| callback_url | No | Callback URL for async processing. If provided, the task runs asynchronously and results are sent to this URL when complete. | |
| expected_type | No | Hint about expected page type. Options: 'product', 'article', 'general'. Helps the extractor optimize for the content structure. | |
| block_resources | No | Resource types to block during page load to speed up rendering. Options: 'image', 'font', 'media', 'stylesheet', 'xhr', 'fetch'. | |
| cache_ttl_seconds | No | Override the global cache TTL (seconds) for this entry. 0 means do not cache this response. Default is 3600. | |
| wait_for_selector | No | CSS selector to wait for before extracting content. |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |