extract_structured
Use this to get a specific data shape from a page using a JSON schema when you can describe the fields but not their selectors - product details, job listings, event data.
Instructions
Use this to get a specific data shape from a page using a JSON schema when you can describe the fields but not their selectors - product details, job listings, event data. Uses an LLM by default; falls back to CSS selectors when no LLM is configured. Not for pages with stable markup whose selectors you know (scrape_structured, no LLM, cheaper). Asking again for the same page with different formats, paths or selectors reuses this session's copy for 10 minutes at 1 credit; pass maxAge: 0 to force a fresh fetch. Cost: 3 credits. Example: extract_structured({url: "https://jobs.example.com/post/123", schema: {properties: {title: {type:"string"}, salary: {type:"string"}}, required:["title"]}})
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to extract structured data from | |
| maxAge | No | Reuse this session's copy of the page if it was fetched within this many ms (default 600000 = 10 min); 0 always fetches fresh. | |
| prompt | No | Natural language instructions for extraction | |
| schema | Yes | JSON schema defining the data structure to extract | |
| llmConfig | No | LLM provider configuration for AI-powered extraction | |
| user_agent | No | Override the outbound User-Agent. CrawlForge identifies itself honestly by default; use this only for targets you have your own agreement with. | |
| selectorHints | No | CSS selector hints to guide extraction | |
| respect_robots | No | Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default. | |
| verify_numbers | No | Numeric provenance guard (default: true): every price or numeric value the LLM returns must appear literally in the page source, else it is returned as null with a reason in `provenance.unverified`. Set false to get the model's raw numbers back, including ones it derived (a count, a sum, a total) rather than read off the page. | |
| fallbackToSelectors | No | Fall back to CSS selector extraction if LLM is unavailable |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| data | No | Extracted fields matching the requested schema; every requested field is present, null when not found | |
| _cost | No | Cost-transparency metadata (D3.5): what this call was projected to cost and what it was charged | |
| cache | No | Page cache: whether this call reused a copy of the page fetched earlier in the session | |
| error | No | ||
| model | No | Model that produced the data, e.g. "gemma3:12b"; present when extraction_method is "llm" | |
| success | No | False when the extraction errored, or a required field came back missing, empty, or in the wrong shape | |
| provider | No | LLM provider that produced the data ("ollama" | "openai" | "anthropic"); present when extraction_method is "llm" | |
| next_step | No | Failed calls only: the tool or fix to try next | |
| confidence | No | 0-1: method and validation, scaled by the share of requested fields filled and, for "llm", the share of short string values found in the page | |
| provenance | No | ||
| validation | No | ||
| schema_used | No | ||
| missing_fields | No | Requested schema fields no extraction method found on the page; each is present in data as null | |
| processingTime | No | ||
| extractionNotes | No | ||
| extraction_method | No | "llm" | "css_fallback" | "keyword_fallback" | "none" |