Scrape a webpage
web_scrapeFetch any URL and return clean content as markdown or structured JSON via CSS selectors. Uses a lightweight HTTP request first, then a browser only when JavaScript rendering is required.
Instructions
Fetch a URL and return clean content — markdown by default, or specific fields via a CSS-selector schema. Tries a plain HTTP fetch first and only launches a browser if the page needs JS to render.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| format | No | markdown | |
| schema | No | field name -> CSS selector, required when format=json | |
| selector | No | scope markdown extraction to this CSS selector subtree | |
| forceBrowser | No | skip the static-fetch tier for pages known to require JS | |
| ignoreRobots | No | ignore robots.txt restriction rules |