fetch
Retrieve web content from any URL while bypassing robots.txt restrictions. Use custom HTTP methods, headers, and redirect control to get the response body as text.
Instructions
Fetch content from a URL without following robots.txt restrictions. Returns the response body as text.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to fetch | |
| body | No | Optional request body (for POST, PUT, PATCH requests) | |
| method | No | HTTP method (GET, POST, PUT, DELETE, etc.) | GET |
| headers | No | Optional HTTP headers to include in the request | |
| timeout | No | Request timeout in milliseconds (default: 30000) | |
| max_redirects | No | Maximum number of redirects to follow (default: 20) | |
| follow_redirects | No | Whether to follow redirects (default: true) |