removedInput schema / properties / allow_subdomains
Removed value: -{
- "type": "boolean"
-}
removedInput schema / properties / content_spec
Removed value: -{
- "description": "Which elements of each page to keep: {preset: 'full'|'main', include_comments: bool, exclude_selectors: [css], include_selectors: [css], keep: {images: bool}}. 'main' = article body only; 'full' = the whole page including comment and discussion threads. Comments, replies and discussion survive only with 'full' plus include_comments; otherwise they are stripped out.",
- "type": "object"
-}
removedInput schema / properties / delay_ms
Removed value: -{
- "description": "Politeness delay between fetches per host (default 250).",
- "type": "integer"
-}
removedInput schema / properties / device
Removed value: -{
- "description": "A linked Writ desktop's agent_id (writ_devices) to act on. Omitted: the desktop this connection chose with writ_devices action='use', if any.",
- "type": "string"
-}
removedInput schema / properties / exclude_paths
Removed value: -{
- "items": {
- "type": "string"
- },
- "type": "array"
-}
changedInput schema / properties / executor / description
Previous value: -"regular (default) = deterministic crawl, no AI. ai = a fleet of AI agents reads every page against `extract_prompt` and returns structured records, for data with no clean CSS selector. Bills at 5x the page rate."New value: +"regular (default, no AI) | ai (5x the page rate)."
changedInput schema / properties / extract_mode / description
Previous value: -"markdown (default) | schema (uniform records via extract_schema) | html (each page's raw HTML, for selectors or embedded JSON)"New value: +"markdown (default) | schema | html."
changedInput schema / properties / extract_prompt / description
Previous value: -"Required with executor=ai: what each agent extracts from each page, in plain language (e.g. 'the product name, price and SKU')."New value: +"With executor=ai: what to extract from each page."
removedInput schema / properties / extract_schema
Removed value: -{
- "type": "object"
-}
removedInput schema / properties / include_paths
Removed value: -{
- "items": {
- "type": "string"
- },
- "type": "array"
-}
changedInput schema / properties / intent / description
Previous value: -"Plain-English goal. The server derives include/exclude paths and a depth from it against a sample of the site's real URLs, and ranks the frontier by relevance, so on an unfamiliar site it scopes better than guessed path regexes."New value: +"The section wanted, in plain words; the server derives paths and depth."
removedInput schema / properties / limit
Removed value: -{
- "description": "Rows of collected data to return when wait=true (default 50).",
- "type": "integer"
-}
removedInput schema / properties / max_age
Removed value: -{
- "description": "Only meaningful with `save_as`: if that saved crawl already completed within this many seconds, return its collected data instead of crawling again. 0 always crawls.",
- "minimum": 0,
- "type": "integer"
-}
removedInput schema / properties / max_concurrent_shards
Removed value: -{
- "description": "Explicit parallel-shard cap; overrides the `speed` allocation.",
- "type": "integer"
-}
removedInput schema / properties / max_depth
Removed value: -{
- "type": "integer"
-}
removedInput schema / properties / name
Removed value: -{
- "type": "string"
-}
removedInput schema / properties / ocr_mode
Removed value: -{
- "description": "auto (default) | off | force",
- "type": "string"
-}
removedInput schema / properties / output
Removed value: -{
- "description": "Response shape, for an answer a program or an API consumes rather than a reader. {shape: 'envelope' (default: Writ's full answer, projected) | 'table' ({columns, rows, total}) | 'records' (bare list of records) | 'record' (the newest record alone: one entity, a usage meter, a dashboard), fields: ['used', 'percent_used as pct', 'items.0.price as first_price'] (ordered pick, renames, dotted paths; a missing path is null, so keys are stable), exclude: ['depth'], include_meta: false (page metadata content_kind/depth/thumbnails is stripped unless true), key: 'usage' (wrap)}. On writ_crawl_site with save_as it is saved as the API's default shape.",
- "properties": {
- "exclude": {
- "items": {
- "type": "string"
- },
- "type": "array"
- },
- "fields": {
- "items": {
- "type": "string"
- },
- "type": "array"
- },
- "include_meta": {
- "type": "boolean"
- },
- "key": {
- "type": "string"
- },
- "shape": {
- "enum": [
- "envelope",
- "table",
- "records",
- "record"
- ],
- "type": "string"
- }
- },
- "type": "object"
-}
addedInput schema / properties / page_budget / description
Added value: +"Cap on pages fetched."
removedInput schema / properties / persona_id / description
Removed value: -"Saved identity to crawl as (writ_personas lists them), for pages behind a login. Every shard shares the persona's signed-in session and one sticky exit IP. 2FA is minted server-side. A desktop persona ('device:…') crawls on its own desktop, from that machine."
removedInput schema / properties / preview_chars
Removed value: -{
- "description": "Cut each inline page's text cells to this many characters (default 12000; 0 = full pages). Cut rows list the fields under `_truncated`; a full page comes from writ_workflow_data(workflow_id=<data_workflow_id>, refs=['<run_id>:<record_index>']).",
- "type": "integer"
-}
changedInput schema / properties / rank_cap / description
Previous value: -"Top N items of a listing (a front page, search results, a category). The server reads the seed page's link order (which is the ranking), seeds exactly those N item pages, and pins the crawl to them, one page per agent, in parallel. Without it the same request is a breadth crawl that mostly collects nav and pagination and does not answer a top-N ask. include_paths, when given, is the item-link shape."New value: +"Top N items of a listing (one page per agent, in parallel)."
removedInput schema / properties / relevance_threshold
Removed value: -{
- "description": "0-1. Every discovered page is scored against `intent` and skipped below the bar, so a broad crawl collects only what the goal needs (≈0.3 for 'the pricing and docs pages'). Unset for a whole-site sweep.",
- "maximum": 1,
- "minimum": 0,
- "type": "number"
-}
removedInput schema / properties / render_mode
Removed value: -{
- "description": "How each page is fetched, independent of `executor`, which decides who reads it. auto (default) = plain HTTP first, warm browser only for JS-challenge or near-empty pages; http = never opens a browser (fastest, static HTML); browser = warm-render every page (JS/SPA sites). executor=ai works on either lane.",
- "type": "string"
-}
removedInput schema / properties / residential_country
Removed value: -{
- "description": "Two-letter ISO country the residential exit should be in (e.g. 'us', 'fr'): pins the exit pool's geo for every shard. Omitted = an automatic exit. Ignored unless the session egresses residential.",
- "type": "string"
-}
removedInput schema / properties / respect_robots
Removed value: -{
- "description": "Honor robots.txt (default true).",
- "type": "boolean"
-}
removedInput schema / properties / same_domain
Removed value: -{
- "type": "boolean"
-}
removedInput schema / properties / save_as / description
Removed value: -"Saves these settings as a named, callable crawl, for one that is re-run later (a recurring pull, an API the user asked for). Saved crawls are listed to every future session as 'already collected', so a saved one-off reads as reusable data to later sessions. Reusing the same name updates that saved crawl instead of creating a duplicate."
changedInput schema / properties / seed_urls / description
Previous value: -"Exact known pages to start from: the crawl collects these instead of discovering its own. The cheapest way to scrape a known set."New value: +"Exact pages to collect (no discovery)."
removedInput schema / properties / shard_size
Removed value: -{
- "description": "URLs fetched per shard batch (default 20).",
- "type": "integer"
-}
removedInput schema / properties / speed
Removed value: -{
- "description": "Throughput tier: slow | normal (default) | fast: the share of the parallel-agent allowance the crawl uses. slow ≈ ¼ at a discounted page rate, normal ≈ ½ at standard rate, fast = all of it at a premium.",
- "type": "string"
-}
removedInput schema / properties / timeout_seconds
Removed value: -{
- "description": "Max seconds to hold when wait=true (≤75).",
- "type": "integer"
-}
removedInput schema / properties / url / description
Removed value: -"Seed URL (required)."
removedInput schema / properties / use_residential
Removed value: -{
- "description": "Route every shard through the platform residential network (premium), for sites that block datacenter IPs / show a bot wall; a persona crawl forces it on automatically. Costs residential bandwidth; default off.",
- "type": "boolean"
-}
changedInput schema / properties / wait / description
Previous value: -"Hold until the crawl converges and return the collected pages in this call (`data.rows`). Default true for a bounded crawl (rank_cap or seed_urls: a few pages, seconds) and false for an open site crawl (returns a crawl id for writ_crawl_status). Past the 75s ceiling the answer is a 504 that still carries the crawl id."New value: +"Hold until done and return the rows (default true when bounded)."