Create extraction
create_extractionCreates a reusable website extraction recipe and starts agent discovery to explore the site, build a scrape plan, and validate records against your schema.
Instructions
create extraction
Create an extraction — a reusable recipe for scraping structured records from a website — and start its agent discovery. Discovery explores the site, builds the scrape/extract plan, and validates it against your schema.
Poll GET /extractions/{id} until status is ready, then execute it with POST /extractions/{id}/runs. The extraction (ext_...) and its runs (run_...) are different resources.
Creating an extraction charges discovery credits (returned as credits_charged). To see the price first, send estimate_cost: true.
Example — Create an extraction: { "starting_url": "https://example.com/products", "goal_description": "Extract every product with its name and price", "output_record_json_schema": { "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "string" } } }, "required_keys": [ "name" ] }
Workflow: Discovery is async but returns no run — poll get_extraction until status is 'ready', then execute with run_extraction.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Optional display name | |
| max_credits | No | Credit ceiling. Refused with 422 credit_cap_exceeded (nothing charged) when the estimate's maximum exceeds it. | |
| starting_url | Yes | Where the agent starts exploring | |
| estimate_cost | No | Validate and price the request without running or charging anything; the 200 response is the credit estimate (EstimateCostResult). | |
| required_keys | No | Record keys that must be non-empty for a record to count | |
| goal_description | Yes | What records to extract, in plain language | |
| output_record_json_schema | Yes | JSON schema of one output record |