fetch_page
Fetch web pages as clean, deterministic markdown with provenance, rendering JavaScript and supporting pagination for long pages.
Instructions
Fetch a web page as clean, deterministic markdown with provenance.
Renders the page in a headless browser (JS/SPA content included), classifies its shape (article / feed / page / app), and emits hierarchically clean markdown. The YAML front-matter carries title, source URL, retrieval timestamp, auth mode, shape verdict with confidence, and a content hash of the body — the same page state always produces the same body, so hashes and diffs are meaningful. Data tables come back as markdown pipe tables.
Browses with the user's Safari cookies by default (logged-in pages render as the user sees them); set private=true for an anonymous fetch. Use wait_selector (a CSS selector) only for SPAs that paint late. Feed/listing pages (HN, news fronts) come back as one linked line per story; article pages as clean prose with inline links.
Long pages paginate: on truncation the tail says which start_char fetches the next slice (served from the capture cache, no re-render).
Optional authenticated backends are never used silently. If native capture is gated on a covered URL, the result explains the available choices. Repeat with backend="opencli" or backend="twitter-cli", or set allow_external_fallback=true to permit the preferred installed backend.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| backend | No | native | |
| private | No | ||
| max_chars | No | ||
| start_char | No | ||
| wait_selector | No | ||
| allow_external_fallback | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |