Web Fetch
web_fetchFetch web page content while bypassing anti-bot blocks using rotating user agents, proxy support, and automatic fallback to Jina Reader for clean markdown.
Instructions
Fetch the contents of a web page with anti-detection capabilities.
This tool replaces the native web_fetch. It uses rotating real-browser user agents, optional proxy (Bright Data or any HTTP/SOCKS5), and automatic fallback to Jina Reader API for sites that block direct access.
Three fetch strategies:
"auto" (default): Tries direct fetch first (with UA rotation + proxy if configured). If blocked (403/429/503/captcha), automatically falls back to Jina Reader.
"direct": Direct HTTP fetch only. Uses rotating browser user agents and your configured proxy. Returns raw HTML.
"jina": Uses Jina Reader API exclusively. Best for JS-heavy sites. Returns clean LLM-ready markdown. Handles rendering server-side.
For fetching multiple URLs at once, use web_bulk_fetch instead — it's faster and uses a single tool call.
Args:
url (string, required): The URL to fetch. Must include https:// or http://.
strategy (string, optional): "auto", "direct", or "jina". Default: "auto".
Returns: Page content (HTML for direct, markdown for Jina), HTTP status, final URL after redirects, and which strategy was used.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full URL to fetch, must include https:// or http:// | |
| strategy | No | Fetch strategy: 'auto' (try direct with proxy+UA rotation, fallback to Jina Reader), 'direct' (only direct fetch with rotating UA + optional proxy), 'jina' (only Jina Reader — best for JS-heavy sites and anti-bot bypassing) | auto |