navigate
Visit a URL and retrieve cleaned page content, removing scripts and SVGs. Set timeouts, wait for selectors or network idle, and receive warnings when bot challenge pages appear.
Instructions
Go to a URL and return the page content, cleaned of / tags.
The response is prefixed with the HTTP status line (e.g. "HTTP 200") so a
404/500 that renders a normal-looking page doesn't get mistaken for success.
timeout_ms overrides the default 30s navigation timeout for slow sites.
If the site fetches its real content (prices, listings) client-side after
the initial paint, pass wait_for_selector (a CSS selector that only
appears once that fetch resolves) or wait_for_idle=True (wait for network
activity to quiet down) so you don't read the page before it's ready.
If the response looks like a Cloudflare/Akamai/CAPTCHA challenge page rather than real content, a "bot-challenge page" warning is prepended. This can happen even on a 200/403 that looks superficially normal.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| timeout_ms | No | ||
| wait_for_idle | No | ||
| wait_for_selector | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |