web_scrape
Fetch any public URL and convert it to clean markdown, HTML, links, or metadata. Uses fast HTTP first, then escalates to a real browser when the page is blocked or JavaScript-rendered, returning only the main content.
Instructions
Scrape a URL into markdown, HTML, links or metadata. Fetches a single public URL and returns clean content in the requested formats (markdown, html, raw_html, links, metadata). The request body IS the ScrapeOption object itself — e.g. {"url": "https://example.com"} — do not wrap it in an extra key. With render=auto the request starts as a fast HTTP fetch and escalates to a real browser when the page is blocked or rendered with JavaScript; backend only pins a specific headless-browser engine for the browser tier and is not a render mode. only_main_content (default true) strips navigation, headers, footers and other boilerplate before conversion. Only public pages are supported; respect each site's terms of use and robots directives. A handful of popular sites (Amazon, Reddit, Yelp, LinkedIn, and others) already have a dedicated, more reliable endpoint elsewhere in this API — a failed scrape against one of them names it.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| scrapeOption | Yes | Scrape options |