Fetch Page Content
infobroker_fetch_pageFetch a URL and extract clean content using multiple renderers. Supports targeted questions, recursive crawling, metadata extraction, and content length control.
Instructions
Fetch a URL and extract clean content via a renderer (Jina Reader by default, with native-HTTP, Wikipedia, Internet Archive, arXiv, and Stack Exchange renderers). Use when you have a URL and need readable text, want to ask the page a question, need the page's last-updated date (detect_date), a bounded same-origin crawl (crawl), or structured metadata (extract). Do NOT use for a general topic search (use infobroker_search_web) or for claim verification across sources (use infobroker_verify_claims). Parameter interactions: question switches the response from the whole page to passages ranked against it, sized by passage_size and capped by max_passages; crawl recursively fetches same-origin pages up to config caps; max_length caps the characters returned (default 50000); extract adds JSON-LD, OpenGraph, and microdata alongside the content; renderer selects the extraction backend — jina needs no API key and native_fetch is the fallback when Jina is throttled. Makes external HTTP calls, truncates very long pages, and needs no API key. Fetched pages are auto-indexed into the knowledge base unless the content policy flags them (see manage_kb), in which case flag mode returns them without storage and block mode refuses them. Returns a JSON envelope prefixed [OK] or [ERROR] with status, provider, results, and meta.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch: a single URL, or up to five URLs fetched in parallel | |
| crawl | No | Bounded same-origin crawl: recursively fetch same-origin pages up to config caps (default off) | |
| extract | No | Return structured metadata (JSON-LD, OpenGraph, microdata) alongside the content (default off) | |
| question | No | Question to extract ranked passages for, instead of returning the whole page | |
| renderer | No | Renderer: jina (default), native_fetch, wikipedia, internet_archive, arxiv, or stack_exchange | |
| max_length | No | Maximum characters to return (default 50000) | |
| detect_date | No | Detect and report the page's last-updated date (default from config) | |
| max_passages | No | Number of passages to return (default from config) | |
| passage_size | No | Target words per passage (default from config) |