webextrator_extract
Extract structured content from a web page by providing a URL. Supports sync or async modes, cookies, custom headers, and optional LLM enrichment.
Instructions
Extract structured content from a web page using the WebExtrator API.
Navigates to the specified URL, renders the page, and extracts structured data
such as product details, article content, or general page information.
Use this when:
- You need to extract structured data from a web page
- You want product details, article content, or general page data
- You need LLM-enhanced semantic normalization of extracted content
Returns:
JSON response containing the extracted structured content.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the web page to extract content from. Required. | |
| mode | No | Processing mode. 'sync' (default) waits for the result; 'async' returns immediately with a task_id to poll via the Tasks API. | |
| delay | No | Extra delay in seconds after page load before extracting. | |
| cookies | No | Cookies to install before navigation. Each cookie is an object with at least 'name' and 'value', plus optional 'domain', 'path', 'expires', 'httpOnly', 'secure', 'sameSite'. Useful for authenticated pages. | |
| headers | No | Extra HTTP headers to include with the page request. | |
| timeout | No | Total timeout in seconds for page load. Default is 30. | |
| enable_llm | No | Enable LLM-based semantic normalization for richer structured output. Default is false. | |
| user_agent | No | Override the User-Agent header for the page request. | |
| wait_until | No | Page load wait condition before extracting. Options: 'load', 'domcontentloaded', 'networkidle', 'commit'. Default is 'networkidle'. | |
| bypass_cache | No | Skip the Redis result cache for this request (still writes the fresh result back). Default is false. | |
| callback_url | No | Callback URL for async processing. If provided, the task runs asynchronously and results are sent to this URL when complete. | |
| expected_type | No | Hint about expected page type. Options: 'product', 'article', 'general'. Helps the extractor optimize for the content structure. | |
| block_resources | No | Resource types to block during page load to speed up rendering. Options: 'image', 'font', 'media', 'stylesheet', 'xhr', 'fetch'. | |
| cache_ttl_seconds | No | Override the global cache TTL (seconds) for this entry. 0 means do not cache this response. Default is 3600. | |
| wait_for_selector | No | CSS selector to wait for before extracting content. |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |