fetch_page
Fetch a single web page and return its content in markdown, text, or HTML. Supports size caps and robots.txt overrides, with HTTP errors returned as status codes.
Instructions
Fetch a single web page and return its content (no JavaScript rendering).
format: 'markdown' (default, compact and readable), 'text' (visible text only),
or 'html' (raw HTML).
max_bytes: cap on returned characters; defaults to the server's SCRAPY_MCP_MAX_BYTES.
Oversized content is truncated and truncated is set to true.
obey_robots: override the server's robots.txt policy for this call only.
An HTTP error status (404, 500, ...) is returned in status, not raised.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| format | No | markdown | |
| max_bytes | No | ||
| obey_robots | No |