scrape_with_actions
Interact with a page before scraping to handle logins, clicks, forms, and dynamic content, enabling data extraction from SPAs and login-gated sites.
Instructions
Use this when you must interact with a page before scraping - login, click buttons, fill forms, scroll, or wait for dynamic content to load - for SPAs, login-gated content, or multi-step flows. Actions: snapshot, wait, click, type, press, scroll, screenshot, executeJavaScript (disabled unless the server runs with ALLOW_JAVASCRIPT_EXECUTION=true; refused on the hosted API), select (dropdowns), hover, navigate. Start a chain with {type:"snapshot"} to list the page's interactive elements, including those in open shadow roots and iframes, with stable refs (@e1, @e2 ... in document order), then target those refs in later actions instead of guessing CSS selectors; navigation invalidates refs, so snapshot again after one. Set browserOptions.consent:"reject" (or "accept") to answer a cookie/consent banner before the first action; it is off by default. Set browserOptions.stealth:true to run the chain in the stealth browser, and browserOptions.engine to pick its engine ("auto" by default - camoufox when it is installed, Chromium otherwise, and the result says which ran). robots.txt is respected on every navigation, and each navigation is checked: a bot wall is reported as blocked with the vendor named, per navigation in navigations and for the final page at the top level. A click or form submit that loads another page is a navigation too: it gets its own navigations entry (trigger names the action type), and finalUrl is where the chain ended. browserOptions.proxyRotation routes a stealth chain through your own proxies. Screenshots are taken by {type:"screenshot"} actions (and on error) and stored as crawlforge://screenshot/{actionId} resources. Not for pages that render without interaction (scrape) and not as the first attempt on a blocked site (stealth_mode operation:"scrape"). Cost: 5 credits. Example: scrape_with_actions({url: "https://app.com/dashboard", actions: [{type:"snapshot"},{type:"type",selector:"@e2",text:"user@a.com"},{type:"click",selector:"@e4"}]})
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| actions | Yes | Browser actions to perform before scraping | |
| formats | No | Output formats for scraped content. For an image, add a {type:"screenshot"} action where you want it taken. | |
| maxRetries | No | Whole-chain retries on failure (0-3). A retry re-navigates to the starting URL and replays every action; each attempt is reported under attempts[]. | |
| redact_pii | No | Redact personal data from the text this call returns, before it reaches your context window. true means the free regex pass over EMAIL, PHONE, FINANCIAL and SECRET. The result carries redaction:{entities,count}. Default: off | |
| formAutoFill | No | Form auto-fill configuration | |
| browserOptions | No | Browser configuration options | |
| respect_robots | No | Respect the target site's robots.txt (default: true). Setting this to false is honoured, returns a warning in the response, and is recorded against your API key — it is your decision, not a silent default. | |
| max_inline_chars | No | Largest result to return inline, in characters of its JSON. Over it, the call returns a preview plus a result_handle for read_result instead of the whole result (default 40,000; env CRAWLFORGE_MAX_INLINE_CHARS) | |
| extractionOptions | No | Content extraction options. selectors results are returned as content.json.extracted, so include "json" in formats when passing selectors — without it the extraction is not part of the response. | |
| screenshotOnError | No | Capture screenshot when an error occurs | |
| continueOnActionError | No | Continue executing actions if one fails | |
| captureIntermediateStates | No | Capture page state after each action: intermediateStates[] gets one entry per action (capturePoint is the action's 1-based position) with the url, title and the requested formats at that point |