Scrape URL and respond with HTML
scrape_url_htmlScrape websites blocked by bot detection, captchas, or geolocation restrictions and retrieve the content as HTML. Optionally interact with the page using browser commands before scraping.
Instructions
Use a URL to scrape a website using the ScrAPI service and retrieve the result as HTML. Use this for scraping website content that is difficult to access because of bot detection, captchas or even geolocation restrictions. The result will be in HTML which is preferable if advanced parsing is required.
BROWSER COMMANDS: You can optionally provide browser commands to interact with the page before scraping (e.g., clicking buttons, filling forms, scrolling). Provide commands as a JSON array string. Available commands:
Click: {"click": "#buttonId"} - Click an element using CSS selector
Input: {"input": {"input[name='email']": "value"}} - Fill an input field
Select: {"select": {"select[name='country']": "USA"}} - Select from dropdown
Scroll: {"scroll": 1000} - Scroll down (negative values scroll up)
Wait: {"wait": 5000} - Wait milliseconds (max 15000)
WaitFor: {"waitfor": "#elementId"} - Wait for element to appear
JavaScript: {"javascript": "console.log('test')"} - Execute custom JS Example: [{"click": "#accept-cookies"}, {"wait": 2000}, {"input": {"input[name='search']": "query"}}]
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| browserCommands | No | Optional JSON array of browser commands to execute before scraping. See tool description for available commands and format. |